Theory & Practice

Replies to This Discussion

Permalink Reply by Ledule Bosch on March 3, 2009 at 10:29am

Hanna,

Thanks for your interesting topic. Its not an easy task to convince Management and Politicians to use your data unless there is enough confidence and understanding on the work that you do.
Influencing Management & political decisions: The ultimate aim of performance monitoring system will be to promote performance-based decision making. To the extend possible, M&E should plan in advance, how management decision making processes will be influenced by performance information. For example budget discussions, programming decisions, evaluation designs/ scopes of work, office retreats, management contracts, and personnel appraisals often benefit from the consideration of performance information.
It is good practice to discuss draft findings with primary stakeholders in order to get feedback, reach joint conclusions and agree on next step. Once these findings are agreed upon they can be communicated to other institutions and other government departments, Ministries etc.
Critical review questions to ask are the following:-Who are the stakeholders to receive reports?
-What type of information should be communicated to stakeholders identified? -For what purpose is the information being communicated to the stakeholders?-What type of response is expected from different stakeholders?
. From a corporate governance point of view, ourselves as M&E officials need to ensure the following: 1)proper use of data/information collected from different data sources for purposes of developing quality reports for stakeholders; 2) proper synthesis of the information/data collected before conclusions are drawn on the progress and lack thereof for the programmes, policies and projects; 3) adequate conversion of data into information where necessary for purposes of further evaluating progress and impacts; 4) Facilitate collation of data from different data sources to inform processes of monitoring and evaluation;
Review questions to ask include: Who will be responsible for data analysis?; Is there enough data to draw conclusions on different programmes?; Can there be simulations drawn out of the collected data for purposes of measuring cause and effect?; Is the type of data (continuous or discrete) collected adequate to measure progress on implementation?; What type of analysis will need to be done on the collected data (qualitative or quantitative) and (primary or secondary). Part of the process of institutionalizing M&E involves identifying key questions for understanding present stakeholder information needs( especially Management and Politicians) in order to create effective Monitoring and Evaluation Information Management systems in the future.

The experience for me previously in working at the highest office of South Africa, being the Presidency, has assisted me to extensively consult with the stakeholders through the Stakeholder Information Needs Analysis. The aim really was to 1) help ensure that outputs of Monitoring and Evaluation are communicated specifically and precisely, so that each stakeholder gets what they need, when and how the need it;
2) identify all stakeholders, including their “stake” or interest, level of commitment and degree of influence;
3) establish a working relationship with relevant colleagues and stakeholders;
4) define, for each stakeholder, the type and level of M&E information they require, the purpose for which they require it, the required timing and frequency of communication, and their preferred medium or format for delivery; 5) seek mechanisms for consulting with stakeholders on their information needs to inform planning and decision-making. The range of outputs, in terms of content can then be produced at different levels of detail, designed in hierarchical manner, so that high-level outputs can be produced as summaries of more detailed reports, in order to avoid the overhead of producing multiple outputs; Alternatives at the level of delivery methods will include written reports in hard copy (e-mail, intranet, website, accessible databases) verbal reports (briefings, presentations, progress meetings), graphical or numerical outputs (tables, charts, posters) etc; For each output, the level of responsibility needs an owner for its production, and the approval authority. It is also wise that this process should assist in identifying those whose contributions will be required, and who will receive the output for information. A RACI Chart is helpful in defining and documenting this (Responsible, Approver, Contributor and Informed);
Your Results-based M&E Framework and performance plans for data collection, verification ,analysis and use should have been communicated to ensure that your role is clearly defined and understood by all.

Letus engage on this topic in more detail as we also learn a lot in the process!! Perhaps you might want to come to South Africa for the Confederations Cup in June or next year for the Soccer World Cup! How about that, and dicuss this face to face!!!!


Regards,
Ledule Bosch
South Africa

• ▶ Reply

Permalink Reply by Sithole Humbe on March 4, 2009 at 7:07am

Thanks my colleques for an interesting topic kickstarting 2009. You know what ! the most difficult and challenging thing in the implementation of IRBM in developing countries comes in the form of lack of accurate data collection methodologies and analysis.

We, as experts in management for development results in our countries lack data collection and analysis skills for reasons best known to us. In my country, the data we provide to senior management and politicians lacks depth/ detail and is at most times relegated because of lack of substance. The method of data collection is not systematic. The data that we collect as advisors of senior management tend to be more and at times irrelevent. The strategies we employ for management to use the data are so diverse but they are countered by attacking strategies on their defensive side. In the meantime, our approach to Politicians and senior management should be encouraging pointing out the merits of using the accurate data we provide them with the help of the clients and stakeholders instead of it being offensive. Practictioners of Mfdr should develop M&E task-teams to assist in the follow-ups on recommended way-fowards to politicians and senior management.

What i have seen during the years behind is that Countries in developing world lacks the much needed CAPACITY in terms of skilled human resource base, machinery, performance indicators and independence on data collection to provide timely accurate data for improved decision making. Most decisions done are not fact based but rather politically acclaimed for the benefit of the few.

• ▶ Reply

Permalink Reply by Hannah J Cooper on March 8, 2009 at 2:43pm

Dear Ledule and Sithole,

Thank you for your excellent observations & comments! And Ledule - thank you for the invitation! I would love to come to South Africa and discuss this face to face over some soccer!

I really agree with both of you and your thoughts on data analysis and collection, also in particular to Sithole’s point on capacity.

One issue that is also a challenge is: Access.

Ledule – it seems that you were in a key position that allowed to insight into the needs of senior management and politicians. Knowing what users want is, as you and Sithole both mentioned one of the most important parts (if not the most important part) in developing a M&E system.

Sometimes, however, in large bureaucracies it can be hard to have access to senior management to find out what their data needs are. Then, when given the opportunity to meet with them, we can find that they themselves are not sure of the types of information that would be useful.

Following this kind of situation, the tendency can be to develop massive data collection systems (which can be expensive and cumbersome) and because there is so much information – senior management is no longer interested in using it because the amount of data is overwhelming.

Do you have any suggestions on how to address these challenges? Are there pre defined types of data that senior management could find more useful that others? Or does that depend on the field/mandate of the ministry? What approaches can be used to find out what senior management wants and needs, especially in ministries where, due to other pressures, RBM is not on the top of the list of priorities?

Very much looking forward to you thoughts!

Hannah

• ▶ Reply

Permalink Reply by Sithole Humbe on March 12, 2009 at 8:25am

Hie Hannah

You have started 2009 year sharp like a razor. My comrade, the practical question you posed to the members might be disguised as a bomb! Anyway, let me try to answer it but diplomatically.

In the first place, senior management and the national executive should have admitted that the adoption of IRBM is the solution to achieve developmental issues in their countries. No way out on this.

The second scenario should see that reforms advised/ recommended by IRBM Consultants are implemented holistically not partially.
3RD scene is the ideal cascading of IRBM principles, concepts and components to the grassroot levels detailing out the benefits of using the model to achieve an integrated development issue .

Im of the opinion that senior management are like chameleons that do not have a distinct form or color when we analyze their needs and wants. In my country for example, the Ministries with capacity to contribute to the Millenium Development Goals and National Development Goals are the ones not implementing IRBM in its entirity due to political pressures and economic challenges. In these circumstances, i will note Education Ministry, Lands and Rural Settlement, Health and Transport. In

The major reason for not putting IRBM on Priority List in certain Ministries is based on their continued resistance to the IRBM Values of corporate governance, accountability, transparency , monitoring and evaluation of implemented programs. These people are fond of implementing activities for the benefit of themselves not linked with national development policy at the expense of taxpayers money.

What i have thought might work is customizing their requirements accordingly and match those requirements to IRBM settings. However, the above approach should be adopted once there is implementation of the following reforms
> organisational culture change( Change business process & procedures in line with client-focusing)
> Top leadership support( i.e. senior management to allow
> participatory leadership in strategic planning and annual targets planning
> developing performance indicators not complicated to the detriment of senior management
> always reminding senior management of the agreed Outcomes and impacts rather than funds and benefits
> Capacity building training workshop- i.e. more interactions with senior management through training workshops, providing them with adequate materials for referencing.

above all Hannah, there is no Ministry to be allowed to operate outside IRBM if the executive of the country has fully supported and approved the adoption of IRBM as a solution to economic development, social development and other development initiatives. The executive assembly and the Economic Development Ministry should instead come up with a Bill or Act/ Policy Document giving power to the Implementation of IRBM as law.

• ▶ Reply

Permalink Reply by James O. Wagala on March 17, 2009 at 2:05pm

Hello Hanna,

The use of information for managing for development results is an issue on which I am particularly passionate. And for many reasons. The major one being that developing countries, especially from Africa, have lagged behind in delivering tangible results to their citizens because many executive decisions are made merely from gut feel and not from a point of information. One is left to wonder why this happens so often while government has all the machinery it needs to collect, collate, analyse and interpret data for purposes of decision-making.

I think that more and more, officers engaged in managing reform programmes are collecting more and more data. The need for decision-making based on hard facts, the growing demand for accountability and results, and the increasing demand of development partners for a demonstration (not just reporting) of results force officers to go out and collect more and more data. What remains to be seen is whether this data is used to make executive decisions. I take the stand that the information generated informs very little or nothing of executive decision-making. If reforms initiatives geared towards institutionalizing RBM (IRBM) are to bear any meaningful fruits, then all decisions have to be based on objective evidence. The centrality of the citizen in IRBM dictates that governments plan their services "outside-in". This cannot be done unless all decisions are based on solid facts obtained from the ground.

Secondly, I smell a dysfunction of IRBM because of interference at the executive level. A lot of good information is collected during the data gathering exercise by officers. I say this information is good because it shows the two sides of the coin: the good and the ugly. However, for reasons best known to then and may be for political expediency, leaders of most government institutions deliberately manipulate this data to reflect impressive results for the sake of their survival. Consequently, the officers who dig deep for valuable information are forced not to hunt for it anymore. In lieu, they sit somewhere and 'manufacture' information because "why labour and it will be changed anyway?"

There is need for officers at operational levels to device strategies to coerce senior management or politicians to use the information provided to them. One of the strategies should be obvious to an average officer managing development results: Provide the right information which will stand the test of time. This will make the leadership develop confidence in the information and always rely on it (or on you) for decision making. Other strategies would involve having the information readily available to the leadership whenever they need it so that they do not have the option to try their gut feel when making decisions. The other would be to outline the impact of both the nice and the unpleasant aspects of the information. By magnifying outstanding performance and positively critiquing the unpleasant feedback so that it reflects more of the lessons learnt than of the failure of the leadership, an officer managing development results can easily win-over the leadership to always use the data they provide. Many other strategies can be thought out during the design and implementation stages of a data mining process.

As managers of development results, we know that the feedback from M&E is highly valuable for learning and continuous improvement of reform programmes. Therefore, the importance of data in informing the progress, design, re-modelling and evaluation of programmes cannot be overemphasized. Let us insist on collecting and disseminating accurate data to our leaders even if they are not willing to use it now. With time they will know the value of it. Moreover, this will make lower level operational decisions to be based on objective evidence and hence be credible both in theory and in practice.

• ▶ Reply

Permalink Reply by Bimerew Alemu on March 18, 2009 at 8:13am

Hello Friends!
I am encouraged to participate on strategies of data. I think data gap is a problem that all of us share in common. And opening discussion on this issue is very important. Thank you for initiating the topic.
In our case the country is highly committed to implement the 8 MDGs. For that standard national survey and different administrative data are collected and used. Since the Government's primary objective is to combat poverty, a standard survey (Household Income Consumption and Expenditure Survey) is conducted every 5 year and data generated. Such standard surveys are conducted by an autonomous Statistical Agency, called Central Statistical Agency.

Thank you
Bimerew Alemu

• ▶ Reply

Permalink Reply by Eva Benita A. Tuzon on March 23, 2009 at 4:21pm

In the Department of Agrarian Reform, being the lead government agency in implementing the Comprehensive Agrarian Reform Program in the Philippines, the DAR had evolved the Agrarian Reform Communities' Levels of Development Assessment tool having six key result areas with 58 indicators which are viewed as reflective of rural development. Since CARP is a social justice program that acquires and distribute agricultural lands to qualified beneficiaries, part of the program is a package of services. The ALDA tool is being used to identify and plan, proposed priority projects based on the needs and capacity of the communities where there are agrarian reform beneficiaries. The results of the assessment serve as information policy that guides DAR, and other stakeholders, including donor agencies, which project to implement. Where there is a good tool to measure, wastage is being avoided. The ALDA tool was developed as an internal assessment tool for DAR which was also accepted by the stakeholders as a very critical planning, M&E tool as well.

Evolving the tool was a long process in DAR and it was also subjected for evaluation [as a tool]. It takes leadership to be able to build a tool and political will to use it with consistency, credibility, and of course a number of resources such as money and development facilitators.

• ▶ Reply

Permalink Reply by Ayouba Moussa on March 24, 2009 at 5:33am

Hannah,

Merci d’avoir accepté d’initier et de conduire les discussions sur ce thème combien important de la valorisation des informations fournies par les dispositifs de suivi et évaluation. Si certains acteurs, notamment les bailleurs de fonds et les Agences de développement, déploient des ressources importantes pour élaborer et mettre en place des dispositifs de S&E, c’est bien parce qu’ils sont convaincus de l’intérêt et le bien fondé d’une information de qualité dans la prise de décisions susceptibles de conduire aux meilleurs résultats pour le développement. Il s’agit pour eux d’améliorer la qualité de la gouvernance. Malheureusement, au niveau de plusieurs dirigeants politiques cette vision n’est pas partagée et les informations produites à grand frais sont superbement ignorées et laissées à la poussière. Un tel état d’esprit n’est pas le fruit du hasard, mais résulte d’un type de comportement qui jure avec les nobles idéaux de développement, d’égalité, de transparence et de bonne gouvernance.

Les praticiens de la Gestion Axée sur le Développement ne doivent pas se laisser décourager par les pratiques en cours dans plusieurs pays, qui doivent justement leur pérennité à l’opacité et la faible performance des systèmes d’information actuellement existants. La capacité des systèmes à générer des informations de qualité me semble être le moyen le plus efficace de susciter l’intérêt pour la mesure des résultats du développement et l’intégration des informations produites pour le prise de décisions.

Quand je vois l’intérêt que suscitent les rapports de suivi de la pauvreté dans mon pays au sein des acteurs de la société civile et dans la presse, je me dis qu’il y a une place pour l’espoir et le changement.

La pression de la société sur les dirigeants politiques va les amener à prendre davantage en compte les préoccupations des populations et à utiliser les outils susceptibles de les aider dans la prise de décisions.

Le développement de dispositifs de S&E de qualité est un processus assez long qui requiert l’engagement et la détermination des praticiens, mais aussi et surtout le soutien des bailleurs de fonds et des partenaires au développement.

Moussa Ayouba
CoP MfDR NIGER

Sorry for the reply in english !

• ▶ Reply

Permalink Reply by Hannah J Cooper on March 24, 2009 at 1:43pm

Thank you all for your comments, thoughts, and feedback! This is turning into quite an interesting discussion!

Will respond in further detail shortly - but in the interim, wanted to provide our anglophone members with a quick translation of M. Moussa Ayouba's reply:

"Hannah,

Thank you for agreeing to initiate and lead discussions on this very important subject of the value of information provided by monitoring and evaluation. While some actors, including donors and development agencies, deploy significant resources to develop and establish mechanisms for M & E, it is because they are convinced of the value and merits of quality information in making decisions that may lead to better development results. It is for them, to help improve the quality of governance. Unfortunately, for some political leaders that vision is not shared and the information produced at great expense are superbly ignored and left in the dust. Such a state of mind is not a coincidence, but is a type of behavior that runs contrary to the noble ideals of development, equality, transparency and good governance.

Practitioners on managing for development should not be discouraged by the practice in several countries, whose very survival is due to the opaque nature and the poor performance of information systems that currently exist. The ability of systems to generate quality information seems to be the most effective way to generate interest for the measurement of results of development and integration of information produced for decision-making.

When I see the interest in the monitoring reports of poverty in my country within the civil society and the press, I say there is a place for hope and change.

Societal pressure on political leaders will lead them to take greater account of the concerns of people and use tools which help in making decisions.

The development of M & E quality is a long process that requires commitment and determination of practitioners, but especially the support of donors and development partners.

Moussa Ayouba
CoP MfDR NIGER"

• ▶ Reply

Permalink Reply by Hannah J Cooper on March 25, 2009 at 10:29am

Colleagues,

I just received and email from Jeffrey Douglas Tshabalala from South Africa - regarding our discussion. He has given me permission to share his response with you:

"The tenets of accountability and transparency define the underpinnings of data collection, analysis and use. In our part of the world, unfortunately, accountability and transparency has not caught on in ways that forces policy makers and politics to seek answers to questions of improved performance and unfettered access to public services. In fact one could be forgiven for concluding that the provision of public services is viewed as a "favor" and not as right. This has to change! One way of contributing to this change is the deliberate strengthening of the voice of the population in demanding accessible, equitable and quality public services. Improving democratic processes and the relevant governance institutions is fundamental in anchoring the two questions suggested above.

Flowing from the above situation will be the ambition by politicians to want to respond to the questions using whatever data or evidence is available. This demand-for-evidence will expose the inadequacies and poor of data and the underlying systems. It may therefore become a strategic idea to " engage" the politicians through chairpersons of Parliamentary Committees for instance on the value of evidence for "managing for development results"!

• ▶ Reply

Permalink Reply by Noah Aguh on March 26, 2009 at 8:38pm

This is a very interesting topic. The crush of the matter (i.e. the lack of use of data in decision making in Africa) from my perspective sterms from the fact that most of the time development projects/programs are implemented without the involvement of data/information specialist. This shortcoming therefore makes it almost impossible for the requisite data to be collected and appropriately used. Also, our culture or upbringing does not encourage record keeping (please, I stand to be corrected) and since data is activity driven, it then becomes almost impossible to collect data when the activity that led to the creation of the data has been completed and no records were kept.
We have to be proactive since we are more or less acting as agents of change. We must first and foremost demonstrate the use of data in our personal lives by being factual (since data is facts about objects). We must also equip ourselves with the necessary strategies/skills for data collection, processing, analysis as well as presentation. Also, we need to help our managers identify what data they need to make them look good to their stakeholders.

• ▶ Reply

Permalink Reply by Nimit Leelasorn on April 8, 2009 at 8:31pm

Dear Hannah,

I have been looking at the your initiated discussion briefly - on and off - during my travels in Asia and Africa, which has very interesting.

I am now back home and have recently submitted paper for Co-MfDR (hosted by Asian Development Bank) annual publication, that may be relevant to your discussion, where I suggest our practical approach based on past projects. I have uploaded it into this discussion as well as on my profile.

I hope it will provide different view point for your discussion.

Nimit
Sydney, Australia

Attachments:

• BMA M_E MIS Development for Develop Plan 6.pdf, 90 KB

• ▶ Reply

• ‹ Previous
• 1
• 2
• Next ›
• Page