CAP-Scan: Review of the Pilots

“Results-based monitoring is still assessed to be the weakest link in the overall national governance systems of many countries”. To address this challenge, the OECD/DAC Cluster E has sponsored a coordinated effort to develop a framework for self-assessment—Managing for Development Results Capacity Scan (MfDR CAP Scan)—that is based on the following key principles: a) demand driven; b) low-cost; c) strategic and broad-based; d) flexible and adaptable to country needs, context and priorities.

The framework has been piloted by the governments of Mauritania (July 2008), Niger (April 2009), and Senegal (June 2009) and recently used by the government of Malawi in the Energy sector. This self-assessment tool is conducted on the basis of the five MfDR pillars: Leadership, Monitoring and Evaluation (M&E), Accountability and Partnerships, Planning and Budgeting, and Statistical Capacity by determining the level of development of its capacities on a scale of four major stages: Awareness, Experimentation, Transition, and Sustainable Implementation.

A review was conducted to identify lessons learned from the application of the CAP-Scan in the pilot countries: what worked well and what did not work at the process, methodology and implementation stages of the CAP-Scan including country ownership for using CAP-Scan, its adaptation to the countries, how it is carried out (participation of stakeholders, decision makers) and utilization of the findings, especially the development of action plans, and follow up to the action plans. The purpose of the review is to improve the tool, methodology and the process.

The assessment is based on a review of documents, reports from the pilots, and interviews with pilot country CAP-Scan teams, the World Bank staff and feedback from the Third Annual Meeting of the African Community of Practice (AfCoP) on MfDR held in Senegal in March 2010. The decision to undertake an early review of the CAP Scan has been motivated by urgency to support achieving the Paris Declaration goal of reducing by one-third the number of countries lacking sound performance assessment frameworks. The full review is now made available to all AfCoP members here.

Key Findings

• The CAP-Scan process in the pilots has demonstrated strong country ownership and has generated solid synergy effects across the government and beyond, allowing for sustained interaction among key institutions (Senegal, Niger).
• The pilots recognized the built-in flexibility to tailor the self assessment to country’s particular needs and have motivated government officials to apply the framework on their own in a cost-effective way in other non-pilot sectors – the energy sector (Malawi) and judiciary sector (Senegal); and efforts are currently underway to adopt the tool to the needs of the decentralized government structures in Senegal.
  The self-assessment should be linked to the country’s development strategy and capacity development programs as a fundamental principle in its design.
• The CAP Scan design should involve policy analysis at the initial stage to determine whether specific programs or expenditures have impact on achieving development goals.
• The formulation, financing and implementation of the Action Plan proved to be the “weakest link” in the CAP Scan methodology and process. The Action Plan is the main deliverable of the CAP Scan self-assessment process. The review, however, found that the focus during the pilot implementation was placed on the self-assessment process per se instead on its outcome. In the articulation of the action plan and its operationalization, the CAP Scan methodology needs to provide clear guidance on how issues of priorities and sequencing should be spelled out and how the Action plan should support strategic development results to be achieved. The guidance needs to focus on: relative importance and urgency of these actions; their feasibility and appropriateness in terms of the available resources, and in terms of the political context and country ownership—differentiating between actions that have sufficient support and those that require further discussion and consensus-building.
• The government expectation has also been that Action plan could serve as a basis for aligning and coordinating donor support for capacity development under government leadership, and that development partners could be more engaged in supporting a cost-effective implementation of the action plans.
• Impact/results of CAP Scan are still to be seen. It is evident from the pilots (esp. Niger and Senegal) that the participatory process bears directly on issues of leadership, governance, accountability and its impact. Yet, follow-up actions like introduction, or deepening of a results-based budgeting process, donor alignment to priorities defined by the CAP Scan, or implementation of a capacity development action plan as a result of the CAP Scan is still to be witnessed - and would be substantial accomplishments. The monitoring of implementation of the Action plan is critical to the impact of the self-assessment.
• The pilot countries recognized that the repeated self-assessments over a longer timeline would provide evidence of the extent to which country MfDR performance has improved, or not.
• Establishing a participatory process to monitor implementation is critical to enhanced accountability of all stakeholders. The pilots have demonstrated that the participation of Non-Government actors, such as, Civil Society, professional associations, and others made the assessments more realistic and credible. These stakeholders helped ‘validate’ the assessment results.
• The CAP Scan implementation in one country has generated demand in other countries through sharing of knowledge and experiences. Information about the CAP Scan initiative and the available technical and financial support were central to generating interest in partner countries. The CAP Scan pilots have demonstrated unexpected, or at least, unplanned synergies with the communities of practice (CoPs) on MfDR that created a virtuous knowledge exchange cycle and demand.
• Financial constraints have impacted some of the outcomes, particularly the lack of resources to support the completion of the action plans and the follow-up activities. The CAP Scan pilots have been financed opportunistically. Different sources of funding were used for individual tasks. A key issue for the future CAP Scan implementation will be the provision of predictable and sustainable funding. Scaling-up use of the CAP Scan methodology will also require sustainable/predictable sources of funding, and dedicated professional and operational support.

Conclusion

Overall, it is evident from the pilots that the CAP Scan framework has considerable potential in supporting countries to develop MfDR capacity. The pilots have helped improve the design and implementation process, and supported an extensive hands-on learning by sponsors and participating countries of the national development process. Each pilot has incorporated lessons learned from the previous ones helping improve both—the framework and the process. The introduction of the sectoral self-assessments in Senegal exposed the critical link between the country’s development objectives and programs and the CAP Scan scope as an integral part of the adaptation to the country context. The most critical steps of the CAP Scan self-assessment—the scoping/adaptation and the action planning revealed weaknesses, many of which were addressed as the piloting progressed. The methodology of prioritization and action planning also needed some improvements, as confirmed by the outputs of the workshops, and the participants’ evaluations and recommendations, and would require further refinement. A complete editorial review could help address issues of terminological consistency, quality design of the CAP Scan toolkit and supporting documentation. Revising the facilitator guide could be useful at all levels of CAP Scan implementation. Reflecting the lessons and experiential knowledge gained through the pilots in the CAP Scan self-assessment (framework and process) could significantly improve its effectiveness.