Council for Higher Education Accreditation
International Seminar
Washington, DC
28 January 2010
Dr. Willie Clarke-Okah & Sir John Daniel
Commonwealth of Learning
Abstract
In countries where quality assurance systems are not well established universities can find it useful to invite external visitation panels to advise them on quality improvement and wider issues. However, these can be costly. Drawing on its experience with organising such panels in Africa the Commonwealth of Learning has developed a Review and Improvement Model that is essentially a guided, do-it-yourself approach to quality assurance that keeps cost to a minimum.
Introduction
Thank you for giving me time to talk about the Review and Improvement Model (RIM) to quality assurance that we have developed at the Commonwealth of Learning. Although I am here describing it, my colleague Dr. Willie Clarke-Okah, COL’s Higher Education Specialist, must take the credit for developing the model.
Let me start with the experiences that gave us the incentive to develop this model.
The University of Ghana International Visitation Panel
In 2007 I was asked to chair an international visitation panel to the University of Ghana, which sought external advice after its reputation was tarnished by serious breaches of examination security. It wanted to re-establish its former national eminence and become a global leader in higher education.
The panel consisted of ten international members from seven countries and six Ghanaians. UNESCO’s Stamenka Uvalić-Trumbić was one of the international members and UNESCO did research in support of our work.
The panel visited the campus for two periods of one week separated by four months and we submitted our final report nine months after starting work. Our remit included academic programmes, governance and administrative structures, infrastructure, and resources.
Both the University and the Government of Ghana found the report valuable and it stimulated major reforms that are still ongoing. However, although the benefits of the process were clear, the cost was high. This was not a model that could be rolled out on any scale in developing countries.
Institutional Trial Quality Audit of the University of South Africa
In 2007 COL also organised a trial quality audit of the University of South Africa (UNISA). Here the context was different. South Africa has well-established quality assurance processes and UNISA was due for an institutional audit by the Higher Education Quality Committee of the South African Council on Higher Education in August 2008. UNISA asked COL to conduct a trial audit in advance of the real thing; and also to advise on how UNISA might implement more effectively its role as a major distance learning institution. It is a mega-university with over 200,000 students.
In UNISA’s case Dr. Willie Clarke-Okah coordinated a panel of seven international members from five countries and one local member. UNESCO also participated in the person of Ms. Zeynep Varoglu. The panel had a preliminary meeting in the UK with UNISA officers and then visited UNISA for a week, publishing its report two months later.
Once again, UNISA found the panel’s report highly beneficial, both as a dry run for the official audit, and for what it learned about creating an institutional culture of distance learning. But although this process was somewhat less costly than the University of Ghana panel it was still an expensive exercise.
Lessons learned
COL decided to distil these two experiences into a model that would yield similar benefits but at a much lower cost. Two lessons emerged clearly from our experience in Ghana and at UNISA.
First, as you all know very well, much of the value in quality assurance comes from self-assessments conducted within the institutions. At the University of Ghana institution-wide departmental self-assessments were conducted only after the panel’s first visit and at its request. UNISA conducted a major self-evaluation exercise before the panel’s visit.
Second, it is essential to have some mechanism that encourages staff across the institution to take quality assurance seriously. In Ghana the publicity surrounding the visits of a high-profile international group helped to do this. At UNISA the upcoming audit by the South African Council on Higher Education, which has a deserved reputation for rigorous analysis and robust recommendations, helped to focus minds.
The COL Review and Improvement Model
This led us to develop a model which, like the visitation panel for the University of Ghana, is primarily intended for the institution’s own use for improvement and capacity building. It can also be used to help meet external quality assurance requirements just as UNISA saw its visiting panel partly as a dry run for the official audit a year later.
The COL RIM model has five steps: initiation; internal staff survey; self-review; verification; and follow-up.
Initiation
The initiation phase aims to determine whether an institution is really ready to go ahead with implementing COL RIM and has the ability to do so. By readiness we mean being prepared to change, to have open discussions about what needs to improve, to let go of old habits and to accept new ways of doing things. In practical terms it means that senior management must be committed to leading and advocating the process and to allocating time and people to it. A COL RIM workshop in Nigeria came up with a list of questions to assess readiness.
While initiation can include a preparation visit from COL, this adds to the cost. Instead we try to bring representatives of institutions that are interested to workshops such as the one in Nigeria. A representative exposed to the COL RIM model in this way can provide institutional colleagues with the information necessary for the initiation process.
If the institution makes a positive assessment of its readiness it then signs a Memorandum of Understanding with COL. The parties agree to the statement that “Quality is an emergent property of an institution’s own systematic review and improvement of its own performance”. The MoU provides for liaison people on each side, principles and timelines for implementation, arrangements for external verification, and cost-sharing.
Institutions using COL RIM are at different stages of development and have their own institutional strategies. The MoU can partly tailor the themes of the process to these.
Four themes are essential: Communication; Needs Orientation; Capacity Building; and Quality Management. Optional themes are Engagement and Innovation/Creativity and institutions may exclude research if they wish.
Staff survey
Once the MoU is signed the next step is a staff survey. Administered to all staff electronically by COL, in partnership with the institution, it asks them to rate a set of quality indicators individually at one of three levels, based on performance descriptors, saying how they perceive practices in their own areas of work. A response rate of 25% is considered adequate. This survey is used by the institution’s management as a key input into the next stage of self review.
Self review is the core of the COL RIM model. It relies on a team of senior staff members to scope and investigate outcomes of the institution, focusing both on good practice and on areas for improvement. It is guided by the six thematic questions about the institution’s results, which are:
- How effectively does the institution communicate with its stakeholders?
- How well does the institution provide the outcomes that its stakeholders need and value?
- How effectively does the institution engage with local and international communities?
- How effective are the institution’s innovative and creative responses to a changing environment?
- How effectively does the institution develop the capacity of its people to provide valued outcomes for stakeholders?
- How well does the institution monitor and improve its performance?
All of the quality outcomes of the process are formative outcomes. It is the most valuable part of the process for the institution.
The exercise produces a report that aims to answer the key thematic questions, making judgements on the evidence and recommendations for improvement. Meeting the timelines for the submission of the report within three months of starting is a critical indicator of the institution’s ability to meet the COL RIM standard and is a pre-condition for the next step of Verification.
Verification
The next step of the model is verification of the self-review, led by an external (lead) verifier and involving a team of internal verifiers made up of people who were not involved in the self-review team. These internal verifiers receive training in quality assurance methods from the Lead Verifier.
The aims of the verification are, first, to verify the rigour of the methods, findings and recommendations of the self review and, in doing so, make a summative judgement about the extent to which the institution can be considered a ‘COL RIM’ verified institution. Possible ratings are ‘not verified’, ‘threshold’ and ‘verified’.
The second aim of the verification is to extend the capacity of the staff in quality assurance through training in methods of scoping, evidence collection and forming judgements.
A third aim is to make additional recommendations for action where appropriate, based on the cause and effect analysis of the verification team.
Follow up
Finally there is a follow up stage where the recommendations made in the self review that have been verified, expanded or modified through the verification exercise are implemented. Institutions are invited to give feedback to COL on the effectiveness of the model. For its part COL reports to stakeholders periodically on the collective outcomes of institutions which have implemented the model, and on refinements to it.
Conclusion
We stress that this is a new model. COL has trialled the model once in 2009 with an institution in the Caribbean. The result was ‘not verified’ and the institution wants to come back and go through the process again in 2011. COL will trial COL RIM twice or three times more in 2010 with institutions in Nigeria and Sri Lanka before making the model widely available to Member States. Already eight institutions have asked to use the model.
With this in mind COL is seeking to build up a pool of verifiers so that a verifier-in-training can accompany the current Lead Verifier on the next visits. The organisations represented here at this seminar are an important source of potential verifiers and I ask you to make this opportunity known.
Let me end by summarising what we are trying to do with COL RIM. Its aim is to help institutions, particularly those that are not yet embedded in effective national quality assurance systems, to develop a quality culture without significant out-of-pocket costs.
So the COL RIM model:
- Combines internal and external quality assurance in a low-cost ‘do-it-yourself’ approach which does not require a panel of external experts
- Develops systemic thinking and organizational learning
- Offers credibility without high-stakes consequences for poor performance
- Focuses on improvement and includes capacity building and developmental support
You are all experienced experts in this field. I hope you will agree that this is a useful contribution to the advancement of quality assurance and capacity building in institutions, particularly in those countries that do not have sophisticated quality assurance structures. This do-it-yourself model will help institutions to develop a culture of quality and to be ready for external quality assessment when it comes.
References
Commonwealth of Learning (2009) Handbook for the Commonwealth of Learning Review and Improvement Model – Making Quality Work in Higher Education, Commonwealth of Learning, Vancouver, 61pp.
University of Ghana (2007) Report of the Visitation Panel to the University of Ghana, Commonwealth of Learning, Vancouver, 128pp.
UNISA (2007) Institutional Trial Audit of the University of South Africa, Commonwealth of Learning, Vancouver, 80pp.