Executive Summary
The Commonwealth of Learning (COL) contracted the writer of this report to evaluate the online course for policy-makers working in open and distance learning (ODL) institutions in the Southern African Development Community (SADC) region. Specifically, the contract required the writer to determine whether the online course met course participants' needs (i.e., increasing their understanding and knowledge of policy development processes) and whether the delivery strategies were efficient and effective. The writer was required to focus on the worthiness of the two aspects of course delivery: that is, online delivery using the Internet (e-mail facility and the CD-ROM) and the one-week face-to-face workshop in Maputo, Mozambique.
The evaluation of the online course covered the management process leading to the development of the course, the use of the South African Institute of Distance Education (SAIDE) Web site, online discussions, course materials, the user-friendliness of the online system, the worthiness of the different types of support systems and the participants' views of the online course. In addition, the evaluation of the face-to-face component covered the quality of the workshop structure, process and materials used; the integration of the workshop with the online course; the value of the workshop for the participants; and the appropriateness of the facilitators.
Subsequently, the evaluation covered the pre-workshop period, the one-week workshop period in Maputo and the post-workshop period. The methods of evaluating the online course included, among others, questionnaires, interviews and analysis of documents, e-mail messages and learning activities. The evaluator had access to the participants' and course facilitators' e-mails, so he was able to monitor the contribution of participants and course facilitators, the quality of the discussions and the pace at which participants progressed in the course. Questionnaires were administered online through e-mails and at the face-to-face workshop in Maputo. The formal and informal interviews were conducted at the workshop in Maputo, as well as by telephone when the need arose.
The online course was well conceived and managed. The layout was simple and easy to follow. The hyperlinks were clear, very directive, simplified and easy for participants to follow. The learning activities were challenging, stimulating and relevant to the social context and work environment of participants in the SADC region. At the same time, the activities depicted real-life situations and attempted to tap, enhance and build on participants' work experience.
Most participants were anxious about the online course prior to its commencement. However, after three weeks of participation the level of anxiety had declined for most of them because they had gained confidence in using the CD-ROM and computers. Most participants' level of knowledge and skills regarding word processing, preparing and sending e-mail messages, browsing for information on the Internet and viewing the CD-ROM ranged from average to slightly above average. Indeed, most participants' computers had Internet connectivity and CD-ROM drives. However, participants who accessed the online course through the Internet complained of their computers' slowness in accessing the SAIDE Web site. Also, participants generally had difficulty opening portable document format (PDF) files and/or sending file attachments through e-mail.
Most participants found the instructions on the CD-ROM about the purpose, content, structure, expected outcomes and mode of delivery of the online course very helpful. The comments they received from colleagues and/or course facilitators were also helpful, encouraging, supportive, constructive and confidence-building. The online system was very user-friendly. However, online discussions were not fully used as expected, despite facilitators' efforts and encouragements. Pertinent issues raised in discussions were often not followed through to their logical conclusion.
Concerning the face-to-face workshop, most participants felt that facilitators clearly stated the workshop objectives. The quality of the content and the extent to which facilitators presented it was quite good. The workshop's physical conditions and logistical arrangements were also good. The workshop activities met participants' expectations. The time allocated to the various workshop activities was adequate, although some participants felt that the workshop session on budgeting for an ODL programme was inadequately covered and recommended that more time should be spent on it. The quality of the handouts and audiovisuals was good. The pace of the facilitators' presentations was satisfactory in the sense that it was neither too fast nor too slow. The workshop activities were reasonable: they were neither very demanding nor very light. The time provided by facilitators for active participation, practical activities and group discussions was sufficient. Also, the duration of the Maputo workshop was sufficient.
The aspects of the workshop perceived by participants to be most valuable included: learning about the process of developing a vision and policy framework, learning to recognise characteristics of a good vision, learning to develop guidelines for formulating a good policy, learning about financial planning, group activities, critical reflections, the inputs of resource persons, the exercises on policy formulation and development, the video linking vision to policy development, and the exercise on designing and evaluating a policy framework for an actual ODL programme.
Overall, participants were satisfied with the online course and it met their expectations. The course content, handouts, supplementary readings, assigned activities and course delivery format (i.e., online and face-to-face workshop) were critical features to the success of the course. The aspects of the online course participants felt contributed most to their learning were the way the course was designed, the challenging nature of the course, the introductory activities, the face-to-face workshop in Maputo, support provided by facilitators and the quality of reading materials on the CD-ROM. On the other hand, the aspects of the online course that participants felt contributed least to their learning were the failure to send e-mails, which kept bouncing back to the sender, and the lack of online interaction among participants.
Based on participants' and course facilitators' views and feelings as well as the evaluator's observations, it was recommended that the two modes of course delivery (online and face-to-face) be retained. The topic on financial management should be given more time and attention because it did not receive adequate time and attention at the Maputo workshop. People enrolling in this course in future should be computer literate and have access to the Internet and computer facilities. However, should some participants have no access to computers and Internet/e-mail facilities, they will need to be provided with printed materials. Participants who fail to complete pre-workshop activities should not be allowed to attend the face-to-face workshop.
Course facilitators should regularly check to ensure the participants' progress is in accordance with the objectives of the online course. The course should be offered between June and August rather than towards the end of the year when everyone seems to be very busy trying to meet deadlines at the workplace. The use of assigned learning activities as well as templates for writing responses should be retained. Course facilitators and participants should ensure that issues raised in participants' submissions are responded to rather than ignored. The SADC-TCDE should consider commissioning a survey study within the SADC region to establish the level of computer literacy, Internet connectivity and access to computers with Internet connectivity and CD-ROM drives.
C.M. Magagula
![[pdf]](/colweb/jsp/jahia/templates/myjahiasite/corporate_portal_templates/images/icons/standard/pdf.gif){kind=icon}
