IDEAS: Instructional Design for Elearning ApproacheS

A Framework for the Pedagogical Evaluation of eLearning Environments

I came across the following resource which provides a, "A Framework for the Pedagogical Evaluation of eLearning Environments".

The paper uses Chickering & Gamson's (1987) 'Seven Principles of Effective Teaching' as the framework for examining the potential of Virtual Learning Environments to enhance learning. It also includes a questionnaire based on the viable system and conversational models articulated by Britain and Liber (2004).

The document is made available from Eduforge, "an open access collaborative learning and exploratory environment designed for the sharing of ideas, research outcomes, open source educational software, and tools within a community of educators, researchers and developers."

Reviewing Synchronous Learning Objects

In response to Christy's comments regarding the use of LORI in sychronous environments.

The creators of the LORI describe using a convergent participation evaluation model in conjunction with the rating instrument.  Please see;
Nesbit, J., Belfer, K., & Vargo, J. (2002). A convergent participant model for evaluation of learning ob-jects. Canadian Journal of Learning and Technology, 28 (3). Retrieved  from http://www.cjlt.ca/content/vol28.3/nesbit_etal.html

I believe that both could be applicable for digital objects available synchronously as many of the criteria in the LORI would be relevant; feedback adaptation; learning goal alignment, content quality, presentation design.

You might want to adapt the criteria in the LORI to address some of the dynamics that are present during real-time activities, i.e. ability to guage understanding and respond to the needs of learners.

Learner motivations in a business and academic environment may not be the same, but there needs to be a formal reward system in place to recognize those who take part in the review process.  Recognition also makes the activity more credible, because in effect the organization/ insitution is reinforcing the idea that this activity is a valuable undertaking.  In academia, the idea behind engaging in a rigorous evaluation of the quality of learning materials is to promote it as a scholarly activity similar to articles being peer reviewed for a journal.  Having someone use your LO would be equivalent to a citation in an article which may indicate that your resource is of some value in that particular discipline. One way of recognizing people who engage in these activities in a corporate environment would be to provide them with the training and time to develop/ evaluate these materials.

Learning Object Articles

A number of articles relating to Learning Objects have just been published online in the Interdisciplinary Journal of Knowledge and Learning Objects, including:

Interoperability and Learning Objects: An Overview of E-Learning Standardization, by Norm Friesen.

Interactive QuickTime: Developing and Evaluating Multimedia Learning Objects to Enhance Both Face-To-Face and Distance E-Learning Environments, by Thomas Cochrane.

Learning Objects: Using Language Structures to Understand the Transition from Affordance Systems to Intelligent Systems, by Jacques du Plessis.

A Study of the Design and Evaluation of a Learning Object and Implications for Content Development, by Ferdinand Krauss & Mohamed Ally.

The links to the articles have been updated and all work now. Thanks to Keith for letting me know. All articles are in Adobe Acrobat (PDF) format.

Project Open Source Open Access

The Knowledge Media Design Institute (KMDI) at the University of Toronto is part of a graduate studies program that conducts research in all areas of knowledge media design.

KMDI has just announced the launching of Project Open Source Open Access. It is a "cross-divisional, tri-campus initiative to develop a networked community to share knowledge, enhance coordination, increase awareness, and to encourage research and knowledge mobilisation in this area".

As part of this initiative they will be hosting a lecture series beginning February 10th, 2005. The first speaker will be Michael Geist (Canadian Research Chair in Internet and e-Commerce Law). The series will be webcast live and archived using epresence interactive media (a research project of KMDI). The platform includes, "support for video, audio, and slide broadcasting; slide browsing and review; submitting questions, integrated moderated chat, live software demos and the automated creation of event archives". They are working towards an open source release of this system.

Instructional Technology Applications

Here is the link to a presentation I delivered to a group of students taking "Teaching in Higher Education".  It combines a variety of teaching models with examples of instructional technology applications.

Elearning Planning and Management

As part of the graduate course I am taking on Planning and Management in Distance Education and Training I am reading Marc Rosenberg's book on Elearning Strategies for Delivering Knowledge in the Digital Age. In the first few chapters there were two critical points which spoke to me. First, that technology in and of itself will not enhance learning. The case that illustrates that point well, was the adoption of cable TV in the classroom. As a tool, technology has the potential to enhance learning if it is accompanied by the appropriate instructional strategies which are designed to achieve the learning outcome. I am inclined to endorse Elliot Masie's position that the technological implementation is the easy part of the equation and that the most challenging aspect is developing the new models of interaction that will allow us to truly realize the full potential of elearning.

Rosenberg also contends that "without a comprehensive strategic foundation" elearning implementations are prone to failure. I have seen many elearning projects that were not sustainable because the decision-makers did not set out realistic, long-term objectives that were achievable given the environment they were operating in. That's why over the next few months I will be working with our Dept. of Health Policy Management to develop a comprehensive, multi-year strategic plan for their Online Master of Health Sciences in Health Administration program. The plan will include the costing structure and operational requirements for their program as well as evaluation and performance indicators and methodologies. We are hoping this process will become a template for other programs on campus wishing to adopt similar delivery strategies. Stay tuned on this front, as I will post the materials regarding this plan at this site.

So far the book is a good read, but I do take issue with some of the items Rosenberg lists in his "Benefits of Elearning". Keep in mind that he is coming from a corporate training perspective.

1) "Despite outward appearances elearning is often the most cost-effective way to deliver instruction (training) or information."

From a University perspective this is not necessarily the case. While it may reduce the need for classroom/ instructor infrastructure, the following items can dramatically increase the cost of course development and delivery; software license, hardware, maintenance of course and hardware, expertise (technical/ instructional design), opportunity cost of faculty time (subject matter expert), copyright, multimedia production. There is a potential to become cost effective, but only once the initial startup costs are absorbed and the number of students increase beyond a certain point.

2) Universality. "Concern over differences in platforms and operating systems is rapidly fading".

The increasingly proprietary nature of learning management systems and content is a major barrier to the concept of universality. The consulting fees required to enable the type of exchange to occur between different hardware and software systems is prohibitive. Until open standards/ source are more widely adopted we will not be able to easily exchange information between systems and individuals.

3) Scalability. "Programs can move from 10 participants to 100 or even 100,000 participants with little effort or incremental cost (as long as the infrastructure is in place)."

That last part is the key. Few Universities or colleges have the resources to put the infrastructure in place for such a dramatic increase in participants.

Take our costs for example. We are a small unit supporting about 2500 users.

Software (LMS/ OS) $60, 000
Hardware $50, 000

To increase our capacity beyond 6 000 users would require $150, 000!

I firmly believe that institutions need to employ a costing methodology that reflects the true cost of adopting elearning delivery strategies. Bates's presentation, "The impact of e-learning on the university campus: measuring the costs and benefits"" (windows media version) is an excellent reference. Link for alternate formats (search for Tony Bates).

Benefits of Learning Online

Faculty I consult with often want to know how the learner benefits from online instruction. I respond by saying that web-based education has the potential to change the way students learn. Sociology professor Mark Kassop, does a good job of outlining 10 ways in which he believes online learning excels.

Reference:

Mark Kassop "Ten Ways Online Education Matches, or Surpasses, Face-to-Face Learning." The Technology Source, May/June 2003. Available online at http://ts.mivu.org/default.asp?show=article&id=1059

Introduction: Empowering Online Teaching through Faculty Development

Rogers (1995) defined innovation as, “an idea, practice, or object that is perceived as new by an individual or other unit of adoption.” The articles in this collection have been selected to address the question, how can we promote innovation in teaching? The important distinction for the context of our discussion is that ‘perceived as new’ is relative to the traditional educational practices of a University which consist largely of face to face, lecture style classes. If web-based delivery is adopted and used with the existing instructional paradigms, the effect on learning will be negligible and will therefore not represent an innovation in teaching. Ultimately, it is not the technology that matters, but how we design the learning through the technology (Miller, 1996). Advances in teaching may result when technology is used as an opportunity discover new ways for achieving learning outcomes. Web-based education can represent both a pedagogical innovation and a channel of communication capable of engaging the learner. Technology enables the instructor to create a different kind of curriculum and to establish a different relationship with the student. The virtual classroom provides a new environment for human interaction and allows the teacher to design learning which can be more student-centred and collaborative.

According to Rogers (1995), innovation is adopted by members within society at different times and at different rates. A steep learning curve is overcome quickly by early adopters who absorb the new skills and instructional paradigms quickly while late adopters acquire this knowledge over a longer time span. Information seems to flow easily from innovators to early adopters. The barrier for diffusion is identified by the gap that occurs between early adopters and early main-stream faculty. The size of this gap can be attributed to the level of the interconnectedness between departments at a university. Often faculty have connections that do not extend beyond their department. This limits the reciprocal exchange of teaching methods and content. The number of nodes and connections between departments determines the complexity and richness of the network (Rueter, 1997).

Faculty development is often seen as a key enabler of innovation in teaching as it may serve to link successful teaching initiatives and the broad expertise available throughout a university. Such a network would facilitate the sharing of innovative teaching strategies and encourage the exchange of best practices in the development and delivery of online education. These types of initiatives also help to raise the profile of innovative teaching practices, and foster communication and collaboration between developers and programs.

Each of the following article summaries will examine a different aspect of faculty development and outline the strategies or considerations for promoting innovation in teaching. Together they provide the foundation for the design of the teaching online workshop I deliver at the University of Toronto.

Chapter 1: Online Course Design and Faculty Development

Brown, G., Meyers, C.B., Roy, S. (2003). Formal course design and the student experience. JALN 7(3), 66-77. http://www.aln.org/publications/jaln/v7n3/v7n3_myers.asp

The authors examine the collaboration between faculty and professional course designers at Washington State University and the impact that this work had on the student learning experience. The purpose of this study was to examine whether instructors exposed to the faculty development process were more likely to create technology-mediated learning environments which reflected effective teaching principles. Brown et al. (2003) go on to outline which principles of teaching have the greatest impact on learning. Based on their research they determined that the most important quality of good teaching practice is faculty-student interaction. This type of communication is also the basis for employing other principles of good practice. It is essential in facilitating the exchange of diverse points of view, for providing feedback on performance and for communicating high expectations. Regular interaction also provides the structure necessary so that students’ continue to focus their efforts on required tasks. A series of faculty and student surveys focusing on teaching and learning goals, activities and processes was used to systematically evaluate the use and impact of innovative teaching practices. The research findings suggest that faculty development programs which include pedagogical and technology training enhances the opportunities for interactions which lead to improved student learning outcomes.

This article was chosen because it demonstrates that systematic course design improves students’ opportunities for faculty-student interaction, student-student interaction, and other elements associated with best practice. It provides the rationale for why we employ an instructional design template to help faculty clearly articulate the strategies and technologies for online instruction. Engaging in this exercise gives them an opportunity to identify the types and frequency of communication they would like to occur and how these activities help them to achieve the learning outcomes.

Chapter 2: Constructivist training for Online Teachers

Gold, S. (2001). A constructivist approach to online training for online teachers. JALN, (5)1, 36-57. http://www.aln.org/publications/jaln/v5n1/v5n1_gold.asp

Gold examines the pedagogical role of the teacher in web-based learning, focusing on the training required to make the transition from in-class room education to online instruction. The article focuses on a two-week faculty development workshop aimed at preparing educators to effectively teach online. Participants were engaged in different types of collaborative exercises ranging from virtual field trips, to online evaluations, interactive essays, and group projects. To promote innovation in teaching faculty experienced constructivist approaches first-hand in order to better understand the value of learner-centred instruction. Gold also outlines the case for constructivist online education. As knowledge is based on different experiences and interpretations by the learner, technology can be used to help them make meaning of their environment by providing the tools by which they communicate their understanding of the world. Gold ties Piaget's processes for knowledge construction to the affordances of online learning networks. For example, the online curriculum can be used to solicit problems from students to act as a stimulus for learning activities. The content can be modularized so as to scaffold learning. Collaborative projects can engage students in designing authentic tasks and case studies can be presented to challenge misconceptions of particular theories. Gold tested several research hypotheses to determine the effect of the training on the instructors, most specifically their perceptions of what online teaching involved and concluded that workshop was successful in getting educators to re-examine their current methods of instruction. He also found that, the participants shifted towards a more constructivist orientation, seeing online courses as offering more opportunities for student participation. Of interest is the description and findings of the culminating activity where groups of participants wrote an interactive essay.

I have included this article, because I agree with the author that in order to effectively teach online instructors should experience it from a student perspective, otherwise the tendency is for them to map their existing practice onto the web-based medium. These experiences also help to inform them about the potential benefits and drawbacks of this delivery medium. The design of the course exposed faculty to a variety of instructional strategies and technologies and was the same approach and organizational structure we used in the development of our own web-based workshop at UofT.