|Teaching and Learning Forum 2013 [ Refereed papers ]|
Jenni Parker, Dorit Maor and Jan Herrington
Email: firstname.lastname@example.org, email@example.com, firstname.lastname@example.org
A key challenge for university professionals is to identify how to construct more interactive, engaging and student-centred environments that promote 21st century skills and encourage self-directed learning. Existing research suggests the use of real-life tasks supported by new technologies, together with access to the vast array of open educational resources on the Internet, have the potential to improve the quality of online learning. This paper describes how an authentic online professional development course for higher education practitioners was designed and implemented using a learning management system (LMS) and an open companion website. It also briefly discusses how the initial iteration was evaluated and identifies recommendations for improving future iterations of the course.
A qualitative design based research approach was employed to explore possible solutions for designing and implementing effective online higher education courses based on a social constructivist model of learning (cf. Parker, 2011). Design based research, like action research, is accomplished at the coal face, however, it involves an ongoing iterative process to monitor the effectiveness of a specifically designed artifact (Kelly, 2006). Key elements of this approach include: addressing complex problems in collaboration with practitioners, integrating design principles with new technologies to develop practical solutions to the problem and conducting effectiveness evaluations to refine the proposed solution and identify new design principles (Reeves, 2006).
A review of existing research and informal discussions with higher education practitioners suggested teachers needed to experience new learning environments as learners themselves in order to implement changes to their teaching approach (Maor, 1999). Therefore, one potential innovative solution for changing existing online teaching practices was to develop an online course based on authentic learning principles where university professionals were immersed in the pedagogical environment (cf. Parker, 2011).
In this paper we describe how an online professional development course for higher education practitioners based on authentic learning principles (Herrington, Reeves, & Oliver, 2010) was designed and developed to provide university professionals with the opportunity to: experience online learning from a student perspective, learn how to use authentic learning guidelines to design their own real-life learning courses, explore how new technologies could be used as pedagogical tools to support student learning, and use online social media tools to network with their peers. It discusses student and facilitator reflections about the effectiveness of the first implementation of the course, and finally, presents recommendations for improving the effectiveness of the design approach for future iterations of the course.
Rich student-centered learning environments that engage learners in meaningful discourse with their peers (Darabi, Arrastia, Nelson, Cornille, & Liang, 2010; Garrison, Anderson, & Archer, 2000) and require them to solve real world issues using technologies as cognitive tools (Gravemeijer & Cobb, 2006; Herrington et al., 2010; Maor, 2007) can better prepare learners to deal with "the messiness of real-life decision making" (Lombardi, 2007, p. 3) required in the 21st century workplace. To help foster critical 21st century skills, educators should harness the affordances of open web-based delivery (Collins & Halverson, 2009; Lambert & Cuper, 2008) and encourage learners to become "cognizant and literate in Web 2.0 tools" (Levin-Goldberg, 2012, p. 3).
Herrington et al.'s authentic learning design framework (2010, p. 128) was extended to include learning objectives and identify components of the course that need to be situated within a protected environment (for reasons of confidentiality). This extended framework provided overall guidance for the design and implementation of the course (see Figure 1) and was also used as a support resource to assist participants to design their own online course. Herrington et al.'s elements of authentic learning (2010, p. 18) and elements of authentic tasks (2010, pp. 46-48) were used to ensure the course and task design adhered to authentic learning principles.
Figure 1: The extended Herrington et al authentic learning design framework
The course was designed to meet five learning objectives and an overall complex task was developed to enable participants to demonstrate the use of higher level cognitive skills to achieve the learning objectives. Figure 2 explains the relationship between the learning objectives and the course tasks. The overall task required participants to: plan an authentic online course for their area of teaching in higher education, create a detailed course outline and present a video overview of the course to their colleagues. Specific requirements were outlined in the course guide and example documents, readings and tutorials were used to guide the learning.
Figure 2: Relationship between learning objectives and tasks
The course was implemented using a Moodle LMS and an open companion website created on Google Sites (see Figure 3). The LMS acted as the central hub for course announcements and provided a protected environment for the confidential components of the course. The companion website was the primary learning environment and contained detailed task instructions, course content, task and support resources.
Figure 3: Authentic eDesign course structure
A Skype chat group and a Diigo bookmarking library group were created to encourage participants to engage in social and cognitive discourses. Google Docs was used as a collaborative space for students to share their website URL to facilitate peer reviews. Links to web-based tools such as wikis, websites, blogs, videos and podcasts were included to assist students explore how new technologies could be used as cognitive tools to support student learning. Students were also required to create a blog and reflect on their learning throughout the course.
Fourteen higher education practitioners within three Western Australian universities registered for the course. Several people withdrew from the course before completing the week one activities. All cited lack of time due to high workloads as the primary reason for withdrawing from the course: "we are a little under the pump at the moment, I am writing a whole new unit" (Participant AW, email). Six participants from two universities completed the course.
According to Maor and Volet (2007) high dropout rates in online professional development courses is common and attrition rates vary from as low as 13.5% to as high as 75%. Factors such as motivation, readiness to study, technical skills, and lack of time due to workloads, or family commitments are common barriers to completing online courses.
Despite their lack of time due to a variety of reasons such as; taking a new role (Participant MA, email), running an intensive week teaching an MBA unit (Participant GS, email) and teaching an Open University Australia unit that runs back to back with no breaks (Participant EC, email), it was obvious that practitioners that withdrew were keen to learn about authentic pedagogies and new technologies as many requested to be transferred to the second course scheduled to run in April 2012.
The focus of this paper is a preliminary analysis of the data collected from the anonymous online course evaluation conducted at the end of the first iteration of the course and the facilitator's reflections about the course to identify potential improvements for the second iteration of the course.
Five participants completed the online course evaluation questionnaire which included thirty-five closed questions (using a four point scale, see Table 1 below) and two open short answer questions. The initial data analysis indicates practitioners responded positively to this innovative learning approach as all participants agreed the course was a useful professional development opportunity.
|The course context represented the kind of setting where the skill or knowledge would be applied||60%||40%|
|The course environment provided a flexible pathway, where I was able to move around at will||80%||20%|
|The tasks mirrored the kind of activities performed in real-world applications||100%|
|The task was presented as an overarching complex problem||60%||40%|
|The activities required significant investment of my time and intellectual resources||80%||20%|
|I was able to choose information from a variety of inputs, including relevant and irrelevant sources||40%||40%||20%|
|The tasks were ill-defined and open to multiple interpretations||60%||40%|
|The tasks afforded the opportunity to examine the problem from a variety of theoretical and practical||20%||80%|
|I was required to take on diverse roles across different domains of knowledge in order to complete the tasks||20%||60%||20%|
|Task assessment (evaluation) was seamlessly integrated with the major task in a manner that reflected real-world practices||40%||60%|
|The tasks allowed a range and diversity of outcomes open to multiple solutions of an original nature||80%||20%|
|The learning environment provided access to expert skill and opinion||100%|
|The learning environment allowed access to other learners at various stages of expertise||100%|
|I was able to hear and share stories about professional practice||40%||60%|
|I was able to explore issues from different viewpoints||20%||60%||20%|
|I was able to use the learning resources and materials for multiple purposes||100%|
|I was provided with sufficient opportunities to collaborate (rather than simply cooperate) on tasks||20%||60%||20%|
|I was provided with sufficient opportunities to reflect on the course content and my own learning||20%||80%|
|I was required to make decisions about how to complete tasks||100%|
|I was able to move freely in the environment and return to any element to act upon reflection||80%||20%|
|I was able to compare my thoughts and ideas to experts, teachers, guides and/or peers||40%||60%|
|I was able to work in collaborative groups that enabled discussion and social reflection||20%||60%||20%|
|The tasks required me to discuss and articulate my beliefs and growing understanding||100%|
|The environment provided collaborative group spaces and forums that enabled articulation of ideas||40%||60%|
|The environment enabled more knowledgeable learners to assist with coaching||40%||60%|
|The facilitator provided contextual support and guidance||100%|
|The facilitator provided timely and helpful feedback||100%|
|The activities culminated in the creation of a polished product that would be acceptable in the workplace||80%||20%|
|The task enabled me to present my finished product (concepts and ideas) to a public audience||60%||40%|
|The activities allowed for multiple assessment measures||60%||40%|
|I felt comfortable learning in an open environment||20%||60%||20%|
|The technologies I was required to use in the course aided my learning||60%||40%|
|The recommended readings were useful for learning about the concepts covered in the course||60%||40%|
|The technologies used in the course demonstrated some of the ways these tools could be used to assist student learning||80%||20%|
|Overall I thought the course was a useful professional development opportunity||80%||20%|
It is interesting to note that a couple of the participants did not think the tasks were ill-defined and open to multiple interpretations. Each participant produced a course outline tailored to their specific area of teaching and identified appropriate learning and assessment methods and supporting technologies. No two course outlines were the same, and participants identified a wide variety of methods and technologies which indicated the task was open to multiple interpretations. Perhaps they were suggesting that the task was not badly-defined, which is a common misinterpretation of this element.
In response to the first short answer question: What did you think were the strongest aspects of the course? one person responded "I was able to redevelop my unit plan and activities in my online unit as part of the course ... ready for semester one". Another commented on the flexibility of being able to control the pace of their learning "the online aspect of the unit allowed me to complete the tasks at my convenience". Access to new technologies was another positive aspect identified by a couple of participants: "the opportunity to develop my units with more consideration of how technology can support learning" and "appropriate technology choices".
Responses to the second short answer question: What areas do you think could be improved? identified a few areas for improvement. One person stated "the blogging was difficult as I struggled a bit with the purpose" and another advised "3 hours a week was nowhere near enough time to allocate". Participant workloads were also an issue "because I was so busy, I would have liked the course to have one less element to complete - I didn't complete the video (which I feel guilty about)". A constructive suggestion about the use of the Diigo, Skype and Google Docs technologies was offered by another participant "I wonder if these could have been introduced with a brief, specific activity that both familiarise us with the technology and demonstrated its usefulness to our learning".
The facilitator reflections confirmed that most learners struggled to complete the activities within the allocated time frame and that some participants had issues installing the necessary software on their work computers. They also thought a blog was not the best tool to use for participants to reflect on their learning and encourage discourse about the concepts covered in the readings as the time required to setup and learn about blogging left little time for student reflection due to the short duration of the course.
|Authentic tasks||Time allocation insufficient.||Increase the time allocation and reduce content or simplify tasks (e.g. replace overview video with simple feedback screencast). Advise participants to install software prior to course commencement.|
|Task technologies (Skype, Diigo).||Skype - include reading and forum question about social presence. Diigo - encourage participants to comment on readings and add a resource to the Diigo group.|
|Collaboration||No issues identified but limited collaboration required.||Include peer review of analysis worksheet.|
|Reflection||Pre and post course participant surveys.||Use a different tool so participants can refer back to their pre course survey.|
|Blogging purpose not clear and time consuming.||Replace blog with an easy to use tool (e.g., a forum for weekly reflections).|
The preliminary data analysis, discussed in this paper, appears to support Maor and Oliver's conclusions. Most participants agreed this innovative online approach was an effective method of learning that provided them with new skills and ideas that they are keen to explore in their own courses. A full data analysis is yet to be completed; however, it appears that lack of time due to high workloads continues to be a barrier for educator participation in professional development opportunities. If universities wish to improve the quality of existing online courses, further research is needed to identify ways of overcoming this barrier at an administrative level to encourage greater participation.
Collins, A. & Halverson, R. (2009). Rethinking education in the age of digital technology. New York: Teachers College Press.
Darabi, A., Arrastia, M. C., Nelson, D. W., Cornille, T. & Liang, X. (2010). Cognitive presence in asynchronous online learning: A comparison of four discussion strategies. Journal of Computer Assisted Learning, 27(3), 216-227. http://dx.doi.org/10.1111/j.1365-2729.2010.00392.x
Garrison, D. R., Anderson, T. & Archer, W. (2000). Critical Inquiry in a text based environment: Computer referencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. http://dx.doi.org/10.1016/S1096-7516(00)00016-6
Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research. Abingdon, Oxon: Routledge.
Herrington, J., Reeves, T. C. & Oliver, R. (2010). A guide to authentic e-learning. New York: Routledge.
Hodges, C. B. & Repman, J. (2011). Moving outside the LMS: Matching Web 2.0 tools to instructional purpose. EDUCAUSE Learning Initiative, September. http://net.educause.edu/ir/library/pdf/ELIB1103.pdf
Kelly, A. E. (2006). Quality criteria for design research. In J. van den Akker (Ed.), Educational design research (pp. 107-118). Abingdon, Oxon: Routledge.
Lambert, J. & Cuper, P. (2008). Multimedia technologies and familiar spaces: 21st-century teaching for 21st-century learners. Contemporary Issues in Technology and Teacher Education, 8(3), 264-276. http://www.citejournal.org/vol8/iss3/currentpractice/article1.cfm
Lane, L. M. (2008). Toolbox or trap? Course management systems and pedagogy. Educause Quarterly, 31(2), 4-6. http://net.educause.edu/ir/library/pdf/eqm0820.pdf
Levin-Goldberg, J. (2012). Teaching generation techX with the 4Cs: Using technology to integrate 21st century skills. Journal of Instructional Research, 1(1), 56-66. http://cirt.gcu.edu/jir/documents/2012-v1/goldbergpdf
Lombardi, M. M. (2007). Authentic learning for the 21st Century: An overview. ELI White Papers, EDUCAUSE Learning Initiative. http://net.educause.edu/ir/library/pdf/eli3009.pdf
Maor, D. (1999). A teacher professional development program on using a constructivist multimedia learning environment. Learning Environments Research, 2(3), 307-330. http://dx.doi.org/10.1023/A:1009915305353
Maor, D. (2003). Teacher's and students' perspectives on on-line learning in a social constructivist learning environment. Technology, Pedagogy and Education, 12(2), 201-218. http://dx.doi.org/10.1080/14759390300200154
Maor, D. (2007). The cognitive and social processes of university students' online learning. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/maor.pdf
Maor, D. & Volet, S. (2007). Engagement in professional online learning: A situative analysis of media professionals who did not make it. International Journal on E-Learning, 6(1), 95-117. http://www.editlib.org/p/6203
Oliver, R. (2005). Ten more years of educational technologies in education: How far have we travelled? Australian Educational Computing, 20(1), 18-23. http://acce.edu.au/journal/20/1/ten-more-years-educational-technologies-education-how-far-have-we-travelled
Parker, J. (2011). A design-based research approach for creating effective online higher education courses. WAIER Research Forum. Forum presentation. University of Notre Dame. Fremantle, Western Australia. http://www.waier.org.au/forums/2011/abstracts.html#parker
Pedagogy (2010). MoodleDocs, 2.4. [viewed June 2010]. http://docs.moodle.org/en/Pedagogy
Reeves, T. C. (2006). Design research from a technology perspective. In J. van den Akker (Ed.), Design methodology and developmental research in education and training. The Netherlands: Kluwer.
Rotherham, A. J. & Willingham, D. (2010). 21st Century skills: The challenges ahead. Best of Educational Leadership 2009-2010, 67, 16-21. http://www.ascd.org/publications/educational-leadership/summer10/vol67/num10/21st-Century-Skills@-The-Challenges-Ahead.aspx
|Please cite as: Parker, J., Maor, D. & Herrington, J. (2013). Under the hood: How an authentic online course was designed, delivered and evaluated. In Design, develop, evaluate: The core of the learning environment. Proceedings of the 22nd Annual Teaching Learning Forum, 7-8 February 2013. Perth: Murdoch University. http://ctl.curtin.edu.au/professional_development/conferences/tlf/tlf2013/refereed/parker.html|
Copyright 2013 Jenni Parker, Dorit Maor and Jan Herrington. The authors assign to the TL Forum and not for profit educational institutions a non-exclusive licence to reproduce this article for personal use or for institutional teaching and learning purposes, in any format, provided that the article is used and cited in accordance with the usual academic conventions.