Why All This Fuss About Monitoring and Evaluation?

Colleagues, it falls to me to say a few words at the beginning of this important workshop. I shall try to answer the question: Why all this fuss about Monitoring and Evaluation?

I like reading books about development. Recently I have enjoyed my first contact with the writings of William Easterly, a former World Bank economist who tries to look at the world as it really is and not as many development workers, both theoreticians and practitioners, might like it to be. I commend his two most recent books. The first, which contains some fairly serious economics, is entitled The Elusive Quest for Growth: Economists' Adventures and Misadventures in the Tropics. In essence he shows that many economic hypotheses that have long been disproved by the evidence are still being used by development agencies as the basis for their work. His other book, published this year, is The White Man's Burden: Why the West's Efforts to aid the Rest have done so much ill and so little good. As the title suggests, this book is more general and even more robust.

COL's mission is learning for development. According to Easterly, development today must address two tragedies. The first is simply that there are still so many poor people in the world. This tragedy has motivated heads of state and others to make many grand declarations. These include the Millennium Declaration, with its Millennium Development Goals; the Dakar Goals, the UK's Commission on Africa; NEPAD and so on. COL works within the agenda set by these declarations.

The second tragedy, which gets less attention, is that 50 years of effort by aid agencies and the spending of trillions of dollars have done so little to cure the first tragedy. Some countries are actually poorer than they were 50 years ago, but even where there has been progress it has been much less than expected. The goals and target dates that international bodies set for eliminating hunger or achieving universal education have to be pushed into the future with dull regularity as deadlines come around and targets are missed - often by a wide margin.

Easterly's diagnosis of the reasons for the failure of development would be a talk in itself. Basically he ascribes it to the weakness that rich governments and their development agencies have for grand plans. For him the key to progress is to abandon grand plans and to search for local solutions. True development needs more searchers and fewer planners.

He also argues that one reason development efforts have failed for so long is that development agencies are not accountable to those they are trying to help. Rich country taxpayers respond well to grand plans and announcements of large sums of money for development, especially if popular figures like Bono are brought in to promote the package. Much less attention is paid to what actually happens on the ground as a result of all this money and hype.

How does all this relate to our work at COL? First, COL is obviously part of the rich world's aid architecture and our two biggest donors, Canada and the UK operate in the context of the grand plans that Easterly finds unproductive. However, COL also receives financial support from 30 developing countries and has close links with them. That allows us to operate close to the grassroots.

Second, in terms of Easterly's dichotomy between planners and searchers, COL is mostly a searcher. We do our own three-year plan, and I shall come to that in a moment, but our mode of operation is more one of searchers than of planners. That too, makes us more of a grassroots organisation. We have general models in our minds for what we are trying to achieve but we look to apply these in concrete ways at the grassroots level by finding out what works.

This gives us a considerable advantage, compared to the big agencies, when it comes to monitoring and evaluation. They are almost always working with intermediaries who may have a vested interest, if only for the sake of their per diems, in saying that a project is wonderful even if it isn't. COL tends to be closer to the real beneficiaries such as poor people, untrained teachers or villagers threatened by disease. This allows us to get a more authentic view of the impact of what we do. But that does not mean we can take monitoring and evaluation for granted. Let me make a few comments about both elements from the perspective of COL.

To be pragmatic: we are a multilateral intergovernmental body reporting to 53 governments individually as well as to the collectivity of Commonwealth governments through CHOGMs and CCEMs. This means that, at the very least, we need to know at any moment what we have done in each country in the last few years. We need to monitor our work.

Dr Samuel Johnson once said that 'the true art of memory is the art of attention'. He meant that we are unlikely to remember something, like someone's name, unless we actually pay attention to it when we are introduced to them. In the same way, COL will not have a collective memory of its work unless it pays attention to it by recording it as it goes along.

By and large we have made good progress on this through our system of trip report templates. The challenge now is to turn these into progress reports that capture all significant developments in COL's programmes - not just those that involve a visit or an event.

We have also made good progress on using our knowledge management skills and technology to enable us to 'know what COL knows'. Our Country Profiles are constructed using our knowledge management system. I am sure that you have all found them very useful, as I have, in preparing your historical reviews of our country activities.

Concerning evaluation my own bias can be summarised as impact and meaning. By impact, I mean impact for the end user. Did the farmers in the villages involved in L3 Farmers actually get richer? Did the people who saw videos produced with the help of the COLME programme make healthier choices? Do teachers who engage in professional development via ODL become better teachers? Do students who complete the CEMBA programme have better career opportunities?

But for me meaning is even more important because it helps make our programmes better in the future or more widely applicable. Patrick Spaven's evaluation of L3 Farmers helped me to understand the interplay between the different elements in the model better than I had before.

Understanding the elements and processes involved in the models that we use for implementing technology-mediated learning is very important. After all the effort that we have put into preparing the 2006-09 Plan there is a temptation to treat it as a static document. It is not.

As we go into each annual cycle of budgeting and planning within the triennium we need to update our understanding of what is working, well and what is not working well and needs to be abandoned or re-oriented. The evidence of our impact and the improved understanding of our models, gained through monitoring and evaluation, help us to make these annual planning decisions.

As well as implementing our models more effectively in their original environments, monitoring and evaluation also help us determine, in advance, whether conditions are propitious for trying out the model somewhere else. If we understand what the crucial element of the model are, we can check whether they are present in the new environment or whether appropriate substitutes can be found.

All this assumes, of course, that we are operating with some sort of model in mind, or at least a clear goal. I said earlier that at COL we operate more often as searchers than as planners, and this is a virtue. We are searching for opportunities to use technology-mediated learning to help poor people. Because we are opportunistic it is even more important that we think through what we are doing in terms of models so that we can analyse our work, criticise it and improve it.

It has been said that if you don't know where you are going any direction will take you there. In preparing the new three-year plan COL consulted more widely than ever before and commissioned a formal evaluation. This led us to focus more sharply than before and create a logic model that expresses our work in terms of 15 initiatives divided into three sectors.

This framework will also be helpful in assessing impact and developing meaning through monitoring and evaluation. This session will increase our understanding of what we are trying to achieve for people at the grassroots as we grapple with how best to monitor each initiative. Discussing the outputs and outcomes of the initiatives from the perspective of the stakeholders will help us to see where we need greater understanding of the models we are using. All this will contribute to our search for meaning in the use of technology to enhance learning for the purposes of development.

There is a very good line in the article in your binders on measuring the performance of public programmes which says that good performance management is an exercise in storytelling. A well developed performance framework allows you to tell a convincing story backed by credible evidence, about how you have made a difference.

The combination of the plan that we have developed and the framework for gathering evidence that you will construct here will be the basis for the convincing stories that COL can tell about its contribution to development in the coming years. We already have such stories, and I am playing truant from this no-travel period on Friday to go and tell the story about L3 Farmers to the Royal Agricultural Society of the Commonwealth's conference in Calgary. We need more such stories.

It's nice to have good stories for external consumption but I urge you to think of monitoring and evaluation primarily as a way of illuminating and enhancing our work for our own intellectual and moral satisfaction; rather than as a tiresome requirement of external funding bodies.

I believe that the three-year plan on which we are embarking will make it easier to integrate monitoring and evaluation in a natural way. We must also ensure that our knowledge management tools become multi-purpose instruments; minimising the number of forms we have to fill in and maximising the meaning that we can derive from them.

COL has already benefited greatly from the evaluative work of both Patrick Spaven and Glen Farrell. I am grateful to them both for teaming up for this training session and I ask you to give them your undivided attention. You have often been told: "You are frequent flyers and have heard these safety announcements many times before. Nevertheless, each aircraft is slightly different and we ask for your full attention".

I paraphrase that by saying that you have all conducted monitoring and evaluation on previous occasions. However, the 2006-09 Plan is different and gives you a new opportunity to use monitoring and evaluation to your professional advantage. Seize it.