Implementing Learning Design: An ALTC Project

This project initially explored the issues to emerge from the implementation of learning designs and identified the barriers to their widespread adoption and ways of overcoming them. These findings then underpinned the implementation of a learning activity planning tool that addressed these adoption challenges in its design and streamlines the lesson planning process.

Project questionnaire: link to questionnaire

The following questions were addressed by this study:

• What learning designs can be readily adopted by particular disciplines as templates for best practice?
• What pedagogical issues emerge from the implementation of learning designs in particular contexts?
• How can identified barriers to academics’ adoption, adaptation and reuse of learning designs be overcome?
• How can the adoption of effective learning designs be facilitated by the use of supports and scaffolds, such as, a learning activity planning tool?

Background:

The expansion, restructuring and refinancing of the Higher Education sector in recent years has meant that classes are not only larger but quite diversified in terms of student ability, motivation and cultural background (Biggs, 2003). This change has created an atmosphere where some lecturers are rethinking their teaching approaches and are seeking out what is known about facilitating effective learning. This is the challenge this project addressed by implementing learning design scaffolds, ie, a learning activity planning tool.

Learning design for the higher education environment is a complex task, especially in light of the increasing diversity of the student body. Learning materials need to be designed to take advantage of different student ability levels, learning approaches & media, and curriculum developed to support a huge variety of outcomes that are often discipline specific. Learning design is a professional activity for which many of our academic staff is not trained. In this project we implemented a learning activity planning tool that provided comprehensive guidance for academics to assist them in the development of inspiring learning design examples and supportive activities.

Methodology:

The project employed a design-based research methodology (Reeves, Herrington & Oliver, 2005) which involved a flexible, iterative process as follows:

• Analysis of current learning design research literature determining needs and opportunities for application of learning designs in the participating universities, by researchers, educational developers and teaching staff.
• Develop an initial Phase 1 design solution to identified needs for learning design development with a planner and guides to using existing learning designs.
• Implement Phase 1 planning tool and guides in participating universities.
• Evaluate outcomes for staff and students from Phase 1 implementation.
• Review of Phase 1 project outcomes by the whole team. Identify research aims and further design and development for Phase 2. Revision of the theoretical framework.
• Disseminate outcomes through workshops at other universities. Upload to a repository for interested staff to implement the learning designs and planner in collaboration with the project team during Phase 2, to enable wider adoption during this phase.
• Redesign planning tool and guides to using learning designs for Phase 2 based on critical needs from Phase 1 evaluation.
• Implement Phase 2 in the participating universities and provide information and online support for the wider group of interested adopters.
• Evaluate outcomes for staff and students from Phase 2 implementation.
• Review of the project outcomes.
• Further dissemination through workshops offered at a range of universities, conference papers, journal articles and promotion of software tools and guides (available through open source software and creative commons licences) via new and existing repositories.

Methods triangulate both quantitative and qualitative data and spanned 7 universities. A literature review was conducted to form a sound theoretical basis for the research. A detailed mapping and analysis of current models was undertaken and workshops will be run with an inter-disciplinary expert working panel to establish best practice.

Progress to date:

The software for the planning tool has been tested and is now considered quite stable. Key members of the project team have gradually refined the template to a point where a basic structure for adding new exemplar learning design templates has been developed. The software is currently being trialled by 93 different users from 9 countries.

The project combined face-to-face workshops for training and sharing of experiences with an online community incorporating discussion forums and will utilise a number of existing repositories (eg. Carrick Exchange, Technology-Supported Learning Database and LAMS Community). An initial successful presentation and workshop at a conference in Cadiz, Spain in July 2008 was followed up by an international Learning Design conference at Macquarie University and a 3 day meeting in December at Sydney University which involved discussions about a variety of related approaches to the problems being addressed by this project. This community has continued to meet, most recently in July, 2009 in the UK, further consolidating the strong affiliations with Oxford, London Knowledge Lab, and the UK Open University. Most importantly, the number of interested parties continues to grow, most recently with a delegation from Greece.

All pilot testing to date has involved only small groups of lecturers and students. However, their feedback has been instrumental in how the Planner interface was designed and how the content was displayed. Evaluation has resulted in a number of interface improvements. Additionally, extensive peer review has been undertaken by our Reference Group and academics well regarded in the field of Learning Design at both a national and international level. The project’s concepts, premises, methods and early prototypes have been evaluated by researchers from around the world – many of whom are working on similar projects. This feedback has already been employed to make some changes in methodology and approach to the finalisation of the literature review. An external evaluator will be employed in the next stage to formally evaluate the project.

Stage 2 testing will involve larger numbers of students in whole class groups across several disciplines and, as per project timeline.

The community also continues to grow by word of mouth and a video presentation has been prepared so that those who cannot be present at a face-to-face presentation or workshop can learn about the Planner. Any enquiries of this nature are referred to: http://wiki.lamsfoundation.org/display/planner/Activity+Planner This page also includes an 8 minute video that outlines many of the features of the current Planner.

The framework will continue to be refined as the data from the Phase 2 pilot is analysed. Currently the framework is evolving as feedback is received from exemplar demonstrations to the Project Team, Advisors and students.

Further information:

James Dalziel

Professor of Learning Technology and

Director Macquarie E-Learning Centre of Excellence (MELCOE)

Contact: james@melcoe.mq.edu.au

Leanne Cameron

Research Coordinator, MELCOE

Contact: leanne.cameron@mq.edu.au

Renewing the Curriculum to More Effectively Accommodate Clinical Rotation

Traditional medical curriculum approaches to practicum involve all students returning to campus one or two days a week for lectures – however, this approach is poorly suited to modern hospital rotation practices (in which different student groups attend different hospital specialties, e.g. oncology or paediatrics ) and most importantly, all students attend the same lecture on the designated weekly topic, rather than individually focusing their study on topic areas relevant to their immediate hospital experiences. This project sought to renew the curriculum and teaching methods of the crucial latter years of medical training through a flexibly delivered, student-centered approach based on innovative technologies.

This project integrates Learning Design approaches, problem-based learning and best practice in online learning delivery methods to address the challenges of maintaining student engagement with theoretical learning modules whilst the students are undertaking their practicum in hospitals in their 3rd, 4th and 5th year of study in a Bachelor of Medicine degree program. The practicum (also called clinical rotations or attachments) unique difficulties in the delivery of relevant information and just-in-time support, given that students are spread across a range of hospitals, many quite a distance from the University.

Background

Despite the efforts of teaching staff to help students make connections and apply their on-campus learning to life in the hospital, there exists a separation of these two worlds. In part, this is a result of students undertaking their studies as separate subjects with the expectation that, along the way, they would 'discover' the connections between them ( Murray-Harvey & Slee, 2000) . Also contributing to the lack of connectedness is the common view of practising medical practitioners that the important learning about medicine occurs in hospitals, not at the University. Therefore, despite the efforts of University staff to help students make connections and apply their on-campus learning to life in the hospital, there exists a separation between these two worlds. There is extensive literature in other fields that provides evidence that better connections can be forged between these two worlds (Bain, 1999; McNaught, Whithear & Browning,1998) and we have used this research to design learning Modules that demonstrate best practice using a problem-based learning style delivered by online learning methods. Each Module represents approximately 25 hours of educational material and activities (typically five case studies of approximately five hours each) on a single scientific strand topic.

Online learning has been used to deliver a radically re-designed course that provides a learning environment that is student-centered, authentic, problem-based and collaborative. By incorporating readings, lectures and other illustrative materials, case studies and activities within a Learning Activity Management System (LAMS) learning activity sequence students have been able to participate in “real-world” problem-solving activities where there is a heavy emphasis on the application of the skills learnt in their practical experience during their clinical rotation (Johnson & Aragon, 2002). Students are able to select the learning module appropriate to their current practicum specialty rather than attend a lecture on a specialty that has been selected by the course program, which may, or may not, have any relevance to the student’s current hospital experiences. As van Merriënboer, Bastiaens & Hoogveld suggest (2004), when a more project-centered approach is employed it gives students the opportunity to explore authentic scenarios where they can make their own decisions on how to implement their learning. Merrill (2002) agrees learning is promoted when students are engaged in relevant “real world” problems. Taking a more problem-based learning approach provides students with the opportunity to gain theory, content knowledge and comprehension in a more authentic way (Major & Palmer, 2001). In addition, this approach helps students develop advanced cognitive abilities such as critical thinking, problem solving, and communication skills (Barr & Tagg, 1995).

Research Questions

The following questions are being addressed by this study:

• What is considered “best practice” for the teaching of the scientific basis of medicine within the field of medical education?
• What learning designs can be readily adopted by the Faculty of Medicine as a means of delivering best practice?
• How can the curriculum be delivered to facilitate students’ learning in relevant and connected ways when they are out on clinical rotation?
• What learning designs can be readily adopted by other disciplines as templates for best practice?
• How can identified barriers to academics’ adoption, adaptation and reuse of learning designs be overcome to ensure sustainability?

Methodology

The project is employing a design-based research methodology (Reeves, Herrington & Oliver, 2005) which involves a flexible, iterative process as follows:

• The Start Up Phase involved the analysis of current learning design, online learning and problem-based learning research literature to determine the needs and opportunities for application of learning designs, by researchers, educational developers and teaching staff.
• Three initial Modules and documentation were developed consistent with the design solutions identified in Step 1 above and in consultation with the Advisory Partners.
• The Implementation Phase involved trialling the first three modules with the University’s 3rd Year School of Medicine students in 2009.
• The outcomes of the Implementation Phase was evaluated by students and staff, Advisory Partners and peer reviewed.
• The initial three Modules and documentation was revised in light of critical needs from evaluation and peer review and devise development framework.
• Development then began on the next three Modules.
• Outcomes have been disseminated through workshops at other universities and conference presentations. Implement second group of three Modules.
• Project outcomes will be evaluated by an external evaluator, and Modules will be revised in light of evaluation outcomes. Revised Modules will be uploaded to an online library for interested staff to implement the Modules in collaboration with the Advisory Partners.
• Outcomes will once again be reviewed, development framework adjusted accordingly and disseminated through workshops at other universities.
• Review of the project outcomes. Further dissemination will be undertake through conference papers, journal articles and guides (available through open source software and creative commons licences) via new and existing repositories.

Methods will triangulate both quantitative and qualitative data. A literature review has been conducted to form a sound theoretical basis for the research. A detailed mapping and analysis of current models has been undertaken throughout the project and five meetings will be run over the life of the project with our Advisory Partners to establish best practice and undergo evaluation.

Theoretical Underpinnings

Learning Design

Learning Design for the higher education environment is a complex task, especially in light of the increasing diversity of the student body. Learning materials need to be re-designed to take advantage of different student ability levels, learning approaches & media, and curriculum re-developed to support a huge variety of outcomes that are often discipline specific. Learning Design is a professional activity for which many of our academic staff are not trained. In this project we have implemented a number of learning Modules that provide comprehensive guidance for staff authors (medical content experts) that assist them in developing appropriate sequences of learning activities based on template Learning Designs.

Problem-Based Learning (PBL)

PBL is an educational format that is centered around the discussion and learning that emanates from a clinically based problem. It is a method that encourages independent learning and gives students practice in tackling difficult situations and defining their own gaps in understanding in the context of relevant clinical problems, hopefully making it more likely that they will be able to recall the material later in the clinical setting. It is a way of learning which encourages a deeper understanding of the material rather than superficial coverage.

This learning style has been incorporated as a curriculum component in a number of medical schools around the world. The small group setting usually used in PBL encourages students to take an investigative approach and take a detailed look at the issues, concepts and principles exposed by the problem. It encourages the development of skills such as literature retrieval, critical appraisal of available information and the seeking of opinions of peers and specialists. PBL allows students to become more involved in, and responsible for, their own learning, and most students and staff report that this is a highly enjoyable way to learn and teach.

Each problem is intended to encourage the student "to develop an appreciation for the interrelated nature of the physical, biological, and behavioral mechanisms that must be considered with each health problem". By participating in this learning format, students will become proficient in the process of problem analysis, hypothesis generation, and the generation of learning issues that warrant further exploration. Each problem is intended to provoke critical enquiry, encourage independent access to a variety of learning resource materials, and generate small group discussion. The depth and breadth of the discussion on any particular topic will vary, depending on where the students are in their course of study (Queens, 2007)

In the current context, students are spread across different hospitals at different times, and rather than returning to campus for lectures on the scientific basis of medicine (as typically experienced by students in a traditional medical course), students use an adapted online form of Problem-Based Learning to explore the underlying scientific research relevant to their particular clinical rotation.

Student Evaluation

As a key focus of our modules has been to promote a collaborative, student-directed, research oriented approach to learning, evaluations from students have been vital in alerted the project team to changes that have needed to be made. Based on the modules that the students have completed to date, five themes have emerged from student feedback:

• The need for clarity of task and task construction;
• Student enthusiasm for collaborative learning;
• Their recognition of the value of active learning;
• The importance of timely lecturer feedback;
• Recognition of the relevance of the theory when it is matched to their practicum rotation.

The need for clarity of task and task construction

When students are out on practicum, there is often a feeling of isolation and poor task instructions can be very frustrating. Many students commented on the value of clear and concise instructions in all learning activities. They also wanted clearly outlined educational goals and the level of student participation expected explicitly outlined. The students preferred their learning activity modules in small, easily accessible sections with easy access so that revision was straight forward. Students requested alternative activity types and ‘richer’ (e.g. transcripts) resources which they feel might benefit their learning, as well as indicating the need for more guidance in terms of their engagement with reading resources. There were also requests for a choice of activity - but too much choice, which can be confusing and time consuming to work through.

Student enthusiasm for collaborative learning

Students regularly commented that they enjoyed collaborating with their peers. They recognised that the collaboration provided different perspectives and these activities required them to think about the subject material.

They enjoyed the forums where they could bounce ideas off each other but a number of students commented that the forum questions need to be broad in scope so that everyone can contribute something – If they weren't one of the first couple of students to respond to the forum question, they sometimes found that there was nothing left to say as the answers had already been covered by the other students.

Student recognition of the value of active learning

The modules employed a range of learning tasks and utilised a number of activity tools. These include quizzes, votes, surveys, question and answers and multiple choice. Student feedback has been largely positive about those tasks that have promoted ‘active’ learning. They particularly enjoyed researching a particular topic and then either discussing their findings in a forum, or responding to a question posed by the lecturer. In this way the students do their own background research, and then the case information is gradually revealed. This method of instruction was preferred to the standard textbook approach.

Lecturer feedback

Receiving timely ‘expert’ feedback has emerged as an important feature. Students responded positively to activities where feedback by their lecturers is provided once they have responded to a question. Students want to know someone is out there noting their responses. The students generally provided negative feedback for those activities that didn’t provide ‘expert’ feedback. There was a general concern expressed about the accuracy of responses provided by ‘peer feedback’. Students didn’t want to hear what answers could be correct – they wanted to immediately know the correct answer delivered by someone who knows.

Relevance of the theory

The learning activities in the modules often included current case information and multimedia resources. Students appreciated their currency and relevance to their practicum, especially when contrasted the information they find in text books. As one student noted,

“It caused me to actively think about all the different aspects of the patient’s needs and the possible barriers to care that they might face. It was a good stimulator of a real-life prescribing process.”

Conclusion

This project has integrated Learning Design approaches, problem-based learning and best practice in online learning delivery methods to address the challenges of maintaining student engagement whilst the students are undertaking their practicum. The compulsory practicum that medical students undergo presents unique difficulties in the delivery of relevant information and just-in-time support, given that students are spread across a range of hospitals, many of which are quite a distance from the University. The learning activity modules produced in this project have addressed this issue directly and are an innovative approach to curriculum renewal. Additionally, once the designs have been fully developed, the exemplar learning design modules can be stripped of their content and adapted for use by other faculties where students are also faced with similar challenges in their own practicum programs.

References

Bain, J. D. (1999).Introduction (to learning-centred evaluation of innovation in higher education). Higher Education Research & Development. 18, 165-172.

Barr, R., & Tagg, J. (1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27 (6), 12‑25.

Bates, A. W. and Poole, G. (2003). Course development and maintenance. Effective teaching with technology in higher education. San Francisco: Jossey-Bass.

Beck, C. & Kosnik C. (2002). Professors and the Practicum: Involvement of University Faculty in Preservice Practicum Supervision. Journal of Teacher Education 2002; 53(1), 6-19.

Boud, D., Keogh, R., & Walker, D. (Ed.). (1985). Reflection: turning experience into learning. London: Kogan Page.

Brook, C. & Oliver, R. Online learning communities: Investigating a design framework. Australian Journal of Educational Technology. 19(2), pp.139-160.

Cameron, L. (2006). Teaching with technology: Using online chat to promote effective in-class discussions. In Proceedings of the 23rd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education: "Who's Learning? Whose Technology?", 3-6 December, 2006, Sydney.

Daresh, J.C. (1990). 'Learning by doing: research on the Educational Administration Practicum', Journal of Educational Administration, 28(2), 34-47.

Dyson, M. (2005). Australian teacher education: Although reviewed to the eyeballs is there evidence of significant change and where to now? Australian Journal of Teacher Education, 30(1), February, 37-54.

Ehrich, L., Hansford, B. & Tennent, L. (2004). Formal Mentoring Programs in Education and Other Professions: A Review of the Literature. Educational Administration Quarterly, 40; 518

Herrington, A., Herrington, J. & Mantei, J. (2009). Design principles for mobile learning. In J. Herrington, A. Herrington, J. Mantei, I. Olney, & B. Ferry (Eds.), New technologies, new pedagogies: Mobile learning in higher education (pp. 129-138). Wollongong: University of Wollongong. Retrieved from http://ro.uow.edu.au/

Johnson, S. D. and Aragon, S. R. (2002). An instructional strategy framework for online learning environments. In T. M. Egan and S. A. Lyman (Eds.) Proceedings of the Academy of Human Resource Development Conference, Honolulu, HI.

Kajs, L. (2002). Framework for Designing a Mentoring Program for Novice Teachers. Mentoring & Tutoring, 10(1).

Kassop, M. (2003) Ten Ways Online Education Matches, or Surpasses, Face-to-Face Learning. The Technology Source, May/June.

Major, C. H. & Palmer, B. (2001). Assessing the effectiveness of Problem-based Learning in Higher Education: Lessons from the Literature. Academic Exchange Quarterly, 5(1). http://www.rapidintellect.com/AEQweb/mop4spr01.htm

Mayes, T. and Freitas, S. (2001). Stage 2: Review of e-learning theories, frameworks and models from ttp://www.jisc.ac.uk/uploaded_documents/Stage_Learning_Models_(Version_201).pdf.

McNaught, C., Whithear, K., & Browning, G. (1998). Systems not projects: focusing on evaluating overall student experience, rather than isolated innovations. Higher Education Research & Development. 18, 247-259.

Merrill, M.D. (2002). "A pebble-in-the-pond model for instructional design". Performance Improvement 41 (7): 39-44. Available at http://www.ispi.org/pdf/Merrill.pdf

Murray-Harvey, R. & Slee, P. T. (2005). Introducing problem based learning to teacher education: A case study. Journal on Excellence in College Teaching, 16(2) 33-55.

Meyers, J. (1995). A consultation model for school psychological services: Twenty years later. Journal of Educational and Psychological Consultations, 6, 73-81.

Queens University (2007). An introduction to PBL. Retrieved from http://meds.queensu.ca/pbl/pbl_in_practice on 18 April, 2007.

Reeves, T.C., Herrington, J., Oliver, R. (2005) Design Research: A Socially Responsible Approach to Instructional Technology Research in Higher Education. Journal of Computing in Higher Education, Spring 2005, Vol. 16(2), pp. 97-116.

Ryan, N. S., Morse, D. R., & Pascoe, J. (1999). FieldNote: a Handheld Information System for the Field. Paper presented at the TeleGeo'99, 1st International Workshop on TeleGeoProcessing, Lyon.

Sharples, M. (2005). Learning as Conversation: Transforming education in the mobile age. In Proceedings of Conference on Seeing, Understanding, Learning in the Mobile Age (pp. 147-152). Budapest, Hungary.