AfCoP Sourcebook LIVE: Managing for development results within the South African Social Security Agency (SASSA):

Replies to This Discussion

Permalink Reply by Dr. Rosa Muraguri-Mwololo on July 8, 2009 at 1:38pm

Hi Ledule,
This is an interesting discussion. The issue of attribution. How can you say with certainty that the social change happening to your target group is solely because of the SASSA program? Two questions- if your program did not exist tommorrow what would the community lose- or what would happen? Is there compelling evidence to support your claims? At impact level one can only contribute to the welfare of citizens acknowledge there are many other actors working to improve the lives of citizens-but at outcome level you can confidently isolate change that is a consequence of your actions. Prove 'beyond reasonable doubt' could be found here if your M&E systems were strong-

It could be more complex than this - but lets hear what others think.

• ▶ Reply

Permalink Reply by Sithole Humbe on July 9, 2009 at 3:17pm

Bosch

How are you friend and comrade. You know what, im in Pretoria right now. The very unfortunate thing is that i did not asked your address before then because there is lot that i wanted to learn from you face to face.
What i think as a strategy to show the grant's contribution is to have a stand alone M&E running Parallel with other NGO or Political programs and apply the independent benchmark(s) from similar implementing countries like Z\imbabwe to your situation. It is a known principle and practice that Government should not be a competing mood with other private players but where it is found wanting ,the independent contributor covers the gap. The government is aiming at the same impact(s which it might have failed in the first place. Ledule, please Govt should be expected to meet its mandatory services delivery before political promises as soon it is on the driving seat by whatever means.

• ▶ Reply

Permalink Reply by Ledule Bosch on July 13, 2009 at 12:15pm

Dear Rosa,

I am also very keen to hear what others will say. Otherwise I am attaching a file as a reading material to assist other respondents to understand the rationale behind setting up an Agency called SASSA so as to have a more rigourous debates around the subject matter,

Interesting ney? Remember, I am still waiting for photos from IDEAS Conference.

Regards,

Ledule

Dr. Rosa Muraguri-Mwololo said:

Hi Ledule,
This is an interesting discussion. The issue of attribution. How can you say with certainty that the social change happening to your target group is solely because of the SASSA program? Two questions- if your program did not exist tommorrow what would the community lose- or what would happen? Is there compelling evidence to support your claims? At impact level one can only contribute to the welfare of citizens acknowledge there are many other actors working to improve the lives of citizens-but at outcome level you can confidently isolate change that is a consequence of your actions. Prove 'beyond reasonable doubt' could be found here if your M&E systems were strong-

It could be more complex than this - but lets hear what others think.

Attachments:

• End of Term Report 2004 to 2008.pdf, 2.4 MB

• ▶ Reply

Permalink Reply by Hannah J Cooper on July 13, 2009 at 5:19pm

Hi Ledule,

Thanks for posting on such an interesting topic!

Quick question - does the public have access to the findings of your Integrated Results-based Monitoring and Evaluation framework? How to your work to disseminate the information found through your M&E work?

Hope all is well!

Many thanks,

Hannah

• ▶ Reply

Permalink Reply by Tamirat Yacob on July 15, 2009 at 8:55am

Dear Ledule,

Thank you for sharing this experience that you have in the SASSA. Well i have some questions that need your clarification and i do have some oppinions too.

Q1. As i can read from the well writen case you have a results based monitoring and evaluation system. And it seems to me that you are more concentrated on the Monitoring side. Do you have any experience in going beyond the Monitoring phase and learn and integrate the lessons in your future work? I know so many countries do strugle with the E side of M&E. I just wnat to know more about this.
Q2. What are the clear criterion that you use to select your benefiaries? This might have some impact on the outcome of SASSA's efforts.
Q3.Is the M&E system available to all parties/public/?

And my oppinion is that, if you register very corectly and have full information on the status of your beneficaries before you put them in to the programe, then you will be able to do before and after analysis using paired sample t-test to compare or at least to see what your program has brough as an impact on your beneficiaries. And i think this will give you some kind of guidance on your future interventions too.

Thank you for your responses and clarifications.

Best regards,

Tamirat Yacob,

AfCoP CMT

Ethiopia

• ▶ Reply

Permalink Reply by Ledule Bosch on July 21, 2009 at 6:26am

Dear Hanna,

Thanks for the questions posed. The public accesses our Results-based Monitoring & Evaluation Framework via internet as well as government information centres called " One Stop Show Multi-Purose Centres" spread throughout the country, especially in rural areas of South Africa. Hardcopies are also distributed throughout the nine provinces and municipalities through government, NGO's and Civil society organisations.

We do have a Communication and Dissemination strategy which we implemented to roll out the framework. The results of data analysis and key research findings are disseminated to internal and external stakeholders, strategic partners and general public using monthly statistical reports, brochures, flyers, fact sheets, newsletters, and a dedicated M&E web page.

Because data travels, and the journey involves transformation from data to information and knowledge, we developed standardized procedures and strategies for appraising partners, politicians, oversight bodies etc on M&E findings.

We can confidently say that the response has been overwhelming.

Regards,

Ledule

Hannah J Cooper said:

Hi Ledule,

Thanks for posting on such an interesting topic!

Quick question - does the public have access to the findings of your Integrated Results-based Monitoring and Evaluation framework? How to your work to disseminate the information found through your M&E work?

Hope all is well!

Many thanks,

Hannah

• ▶ Reply

Permalink Reply by Zaam Ssali on July 21, 2009 at 10:12am

Dear Ledule,
You have an interesting discussion going here. The challenge you are highlighting is common to all working in the development arena. Increasingly, organisations/agencies/institutions are asked to show results accruing from their work but these are hard questions from experience. Contribution and attribution issues are not very clearly defined.

As we all know, attribution of results to an organisation’s interventions sounds nice but the reality is very different. There are too many contributors for a result to happen many of which are out of a given organisation’s control. Consideration of your organisation as having made a contribution to a desired result is actually more credible than attributing the result to your organisation alone.

The real issue would be to show the cause and effect relationships between the results/impact/outcome on the ground to your work as an organisation. The options in your case could be as follows:

1. You mention that a Monitoring and Evaluation Framework was developed for the SASSA. Was a theory of change (ToC) prepared at that stage? This would help the organisation think thru what they expect to happen when the implementation of program/project begins. The ToC can therefore be useful at the evaluation stage when comparisons are made to what was predicted at the beginning and what has happened. The variances from the analysis will aid in attributing the results to your organisation or not. SASSA can consider itself successful if the analysis presents strong indications that the evaluation results are consistence with your ToC.

2. It is not clear from your presentation whether all regions started receiving grants at the same time or if the process was in phases. The option in this case would be, to evaluate and compare the various regions depending on who received first, who received later or who has not received at all. The comparisons may tell an interesting story too. Definitely, the ethical considerations for this option are important to note because you may be accused of having done the phasing intentionally. This implies that you may have denied one region for your ‘random controls’; as you may be aware the debates surrounding this issue are still on-going.

3. What are the feedback mechanisms in place between SASSA and its grantees? A proactive mechanism could be initiated so that you continue to receive information where grantees are able to provide insights into what they think works, what does not work and what explains these occurrences. You may not get all the answers you expect but it may be useful.

4. Initiating good working relationships with other actors in the sector may also be useful. Joint evaluations could be carried out by various actors and sharing information becomes possible so that better analyses are made and hopefully a better understanding of what your contribution is.

5. As we have previously discussed on these pages, communication in MfDR is very important. The tops managers should be made to understand that attribution of results to a given organisation is very difficult; contribution is the way to go. Evidence should be given to the board, top management that the work of SASSA is contributing to change which I believe will prove very encouraging in the long-run

In conclusion, if SASSA’s vision is to improve the lives of the population, contributing to success is enough rather than sorting thru who has achieved what. I also think if an organisation only invests in M&E to prove that their work caused a change then the times ahead could be more challenging. In any case the options above may sound easy but deliberate initiatives in the right direction will be very rewarding though it may take some time.

Best wishes in the M&E efforts at SASSA,

Zaam

• ▶ Reply

Permalink Reply by Ledule Bosch on July 21, 2009 at 10:18am

Dear Tamirat,

How are you. Good to hear from you. Before answering your questions, can you tell me what is it that you are holding with your hands in picture? thought it was an "M&E OLYMPIC MEDAL"

On a serious note, the answers to your questions are as follows:
Q1: We are talking results management and therefore are now focusing on the demand side of our M&E System, which is Evaluation. We have allready rolled out our Multi-Year Evaluation Plan 2007-2011. The period of our plan adheres to government priorities of the Medium Term Strategic Framework.

Q2. Our Social Assistance Programme has clear set of selection criterion. Our M&E System evaluates the targeting of each programme, distinguish between targeting criteria (intended beneficiaries) and targeting outcomes (actual beneficiaries). With respect to targeting criteria, our analysis describe the explicit or implicit targeting goals of the programme and the intended beneficiary groups, the targeting mechanisms used, including formal entry and exit criteria (individual assessment) self-selection mechanisms and or/resource allocation criteria (e.g allocations based on poverty maps), analysis of targeting outcomes using available data to assess the characteristics of actual beneficiaries.Our evaluation also measure leakage to non-intended beneficiaries (errors of inclusion) to capture target efficiency.
Q3. The M&E System is available to the public, government and oversight bodies.

We are doing our best to make sure that full information about our beneficiaries is collected before they enter the program.

Regards,

Ledule

Tamirat Yacob said:

Dear Ledule,

Thank you for sharing this experience that you have in the SASSA. Well i have some questions that need your clarification and i do have some oppinions too.

Q1. As i can read from the well writen case you have a results based monitoring and evaluation system. And it seems to me that you are more concentrated on the Monitoring side. Do you have any experience in going beyond the Monitoring phase and learn and integrate the lessons in your future work? I know so many countries do strugle with the E side of M&E. I just wnat to know more about this.
Q2. What are the clear criterion that you use to select your benefiaries? This might have some impact on the outcome of SASSA's efforts.
Q3.Is the M&E system available to all parties/public/?

And my oppinion is that, if you register very corectly and have full information on the status of your beneficaries before you put them in to the programe, then you will be able to do before and after analysis using paired sample t-test to compare or at least to see what your program has brough as an impact on your beneficiaries. And i think this will give you some kind of guidance on your future interventions too.

Thank you for your responses and clarifications.

Best regards,

Tamirat Yacob,

AfCoP CMT

Ethiopia

• ▶ Reply

Permalink Reply by Ledule Bosch on July 23, 2009 at 5:44am

Zaam Ssali said:

Dear Ledule,
You have an interesting discussion going here. The challenge you are highlighting is common to all working in the development arena. Increasingly, organisations/agencies/institutions are asked to show results accruing from their work but these are hard questions from experience. Contribution and attribution issues are not very clearly defined.

As we all know, attribution of results to an organisation’s interventions sounds nice but the reality is very different. There are too many contributors for a result to happen many of which are out of a given organisation’s control. Consideration of your organisation as having made a contribution to a desired result is actually more credible than attributing the result to your organisation alone.

The real issue would be to show the cause and effect relationships between the results/impact/outcome on the ground to your work as an organisation. The options in your case could be as follows:

1. You mention that a Monitoring and Evaluation Framework was developed for the SASSA. Was a theory of change (ToC) prepared at that stage? This would help the organisation think thru what they expect to happen when the implementation of program/project begins. The ToC can therefore be useful at the evaluation stage when comparisons are made to what was predicted at the beginning and what has happened. The variances from the analysis will aid in attributing the results to your organisation or not. SASSA can consider itself successful if the analysis presents strong indications that the evaluation results are consistence with your ToC.

2. It is not clear from your presentation whether all regions started receiving grants at the same time or if the process was in phases. The option in this case would be, to evaluate and compare the various regions depending on who received first, who received later or who has not received at all. The comparisons may tell an interesting story too. Definitely, the ethical considerations for this option are important to note because you may be accused of having done the phasing intentionally. This implies that you may have denied one region for your ‘random controls’; as you may be aware the debates surrounding this issue are still on-going.

3. What are the feedback mechanisms in place between SASSA and its grantees? A proactive mechanism could be initiated so that you continue to receive information where grantees are able to provide insights into what they think works, what does not work and what explains these occurrences. You may not get all the answers you expect but it may be useful.

4. Initiating good working relationships with other actors in the sector may also be useful. Joint evaluations could be carried out by various actors and sharing information becomes possible so that better analyses are made and hopefully a better understanding of what your contribution is.

5. As we have previously discussed on these pages, communication in MfDR is very important. The tops managers should be made to understand that attribution of results to a given organisation is very difficult; contribution is the way to go. Evidence should be given to the board, top management that the work of SASSA is contributing to change which I believe will prove very encouraging in the long-run

In conclusion, if SASSA’s vision is to improve the lives of the population, contributing to success is enough rather than sorting thru who has achieved what. I also think if an organisation only invests in M&E to prove that their work caused a change then the times ahead could be more challenging. In any case the options above may sound easy but deliberate initiatives in the right direction will be very rewarding though it may take some time.

Best wishes in the M&E efforts at SASSA,

Zaam

Dear Zaam,

How are you doing Zaam? I hope you travelled well home after an exciting IDEAS conference here in Johannesburg, South Africa. Remember, I am still waiting for those wonderful pictures you took, I think its good that we share them with members.

Thank you for your elaborate comments, inputs as well as questions posed.
1. In October 2002 Cabinet of South Africa approved the centralisation of the social assistance grants function, which at the time was a provincial responsibility, through the establishment of a new dedicated national agency, the South African Social Security Agency (SASSA), in 2006. So a theory of change was prepared long before the establishment of SASSA.

2. The rationale for the centralisation of the function and the establishment of SASSA was driven by the need to improve the effectiveness of the administration of social assistance grants, generating efficiency gains, significantly reducing fraud, and ultimately, to serve as the agent for the prospective administration and payment of social security. The major focus was to start with the poorest regions of South Africa where poverty and vulnerability was the order of the day. But this did not deter other provinces from applying for grants based on the fact that it is a constitutional right for the poor and vulnerable to have access to the social safety net.

Since the establishment of SASSA in 2006, the focus has been on transferring the social assistance function from the provinces to SASSA and ensuring uniformity and standardisation of these services across the country. At the same time Government has signalled its intent to create and ensure the provision of comprehensive social security services to mitigate against poverty and vulnerability in South Africa. The investigations into the establishment of the comprehensive social security system has now led to a debate about SASSA broadening its present mandate of administering social assistance and taking on the responsibility of administering other parts of the social security system as well. The need has therefore emerged to review the current institutional, organisational and service delivery models available to SASSA in order to give effect to this broader mandate within the emerging social security reforms in South Africa.
South Africa has a less evolved social security system which is very fragmented and in some instances, certain components are even non-existent. South Africa’s social assistance and social insurance elements focus on paying benefits to beneficiaries who qualify in terms of the eligibility criteria. Thus, South Africa has one on the largest transfer programmes in the world, but the system is by no means comprehensive, integrated or holistic and very far from offering developmental services to people in need

This is not the case in South Africa, where the benefits represent safety nets for those people who are not part of the economically active labour market population, namely children under the age of 15, pensioners and people with disabilities.

Effective administration of the social assistance programs is a core function of the South African government and represents a national priority. The Social Sector Cluster of Cabinet has identified the payment and administration of social assistance as a priority of the government. The establishment of the South African Social Security Agency (SASSA) was based on the myriad inefficiencies and service delivery backlogs that existed when the function was performed across the 9 autonomous provincial departments. As an emerging Agency, SASSA is not only faced with the challenge of unifying the implementation of the social assistance legislation and policies across the nine regions, but is also faced with the challenge of reconfiguring and transforming its benefits administration procedures, processes, systems and operational design to ensure optimal service delivery. Centralisation of the services under SASSA has resulted in homogenized services to a large degree, efficiencies in terms of payments, improved controls with special emphasis on fraud identification and elimination.

The Agency establishment initiative however, should be understood within the context of government’s longer term policy on the establishment of a comprehensive social security system. This would result in the incorporation of not only the functions of social assistance administration and payment, but also the payment of unemployment benefits, workman’s compensation, social health insurance benefits, housing and national student financial aid benefits, periodic payments on behalf of the Motor Vehicle Accident Fund, as well as the administration of a national contributory social pension scheme. The basis for these stems from the Taylor Report on a Comprehensive Social Security System and thus represents the longer term objectives of government.

The focus of reforms in South Africa has now shifted to these contributory schemes as well as retirement provisions with further restructuring of the institutional arrangements inevitably moulding the future of all these entities. The challenge for SASSA and the Department of Social Development, now, is to find an institutional model that can deliver on the envisaged comprehensive integrated and holistic social security system. However, the vision for these institutions needs to be determined now so that both institutions can strategically position themselves.

3. The feedback mechanisms between SASSA and its beneficiaries is done through various kinds such as
3.1 Beneficiary report Cards;
3.2 Annual Beneficiary Satisfaction Surveys
3.3 Presidential and Ministerial “Imbizo’s” meaning gatherings for feedback on service delivery issues by politicians, government and the public, including beneficiaries

4. South Africa has a Country Partnership Strategy with various partners including the World Bank. The Social Safety Net and the Evaluations Component of the World Bank have been key players in technical assistance, training etc. And have undertaken numerous impact evaluation studies around our Social Assistance Programme. The World Bank Country office here in South Africa has also been very helpful in the area of Monitoring and Evaluation.

5. Buy- in and the use of evaluation findings by top management of SASSA still remains a challenge. Decisions are still based on anecdotes and not yet informed by empirical evidence. We are working on that.

I have downloaded some files to get more information. Hope it assists.

Thank you very much for this wonderful contribution from you ZAAM.

Ledule Bosch

• ▶ Reply

Permalink Reply by Ledule Bosch on July 24, 2009 at 8:09am

Additional reading material from SASSA

Attachments:

• Demographic profiles of South Africa.doc, 445 KB
• Social Assistance Programme in South Africa.ppt, 368 KB

• ▶ Reply

Permalink Reply by Zaam Ssali on July 27, 2009 at 5:07am

Dear Ledule,

My dear hope you are well too. Rosa took the pictures at the IDEAS meeting, we should remind her to upload them on the AfCoP website. Thanks for the insight into SASSA's work, provides a more clearer understanding about the programme.

regards,

Zaam

• ▶ Reply

Permalink Reply by Ledule Bosch on July 27, 2009 at 11:41am

Dear Zaam,

Your input and comments are very encouraging, and thank you very much for this. I also believe that it will be in the best interest of everybody participating in this important live discussion that we share with them lessons from the Johannesburg, South Africa's IDEAS conference "Training Program 2009 on Results Based Management for public sector performance. The video clip is attached. Our Framework was enhanced also as a result of such learnings.

Regards,

Ledule

Zaam Ssali said:

Dear Ledule,

My dear hope you are well too. Rosa took the pictures at the IDEAS meeting, we should remind her to upload them on the AfCoP website. Thanks for the insight into SASSA's work, provides a more clearer understanding about the programme.

regards,

Zaam

Attachments:

• Results based management.rar, 45.2 MB

• ▶ Reply

• ‹ Previous
• 1
• 2
• 3
• 4
• Next ›
• Page