Public Service Commission Monitoring and Evaluation System Evaluation Framework

Context/ Constitutional Value

A high standard of professional ethics must be promoted and maintained

Performance indicator

Cases of misconduct are dealt with effectively and in good time.

Standards

1. A system is in place for identifying, recording, managing, and reporting cases.
2. Cases are responded to promptly and finalized.
3. Cases are appropriately resolved.

Underlying assumptions:

Departments that deal with cases of misconduct effectively are generally maintaining a higher standard of ethics than those that do not.

1. Evidence that a system is in place for cataloguing cases as they arise and for facilitating their movement through a given procedure through which they can be tracked.
2. Evidence that cases of misconduct are reported upon in management reports.
3. The length of time taken to process cases should be adequate for due process while avoiding delays that would deny justice.
4. A review should be undertaken of the outcomes of cases.

Format for presentation of quantitative data

Data summary table showing all cases in the reporting period with the following information: transgression type; number of days; case outcome.

Data should also be provided on the number of pending cases and the percentage of cases that were taken on appeal.

Qualitative analysis

An analysis should be undertaken of the reasons for cases and their outcomes. Analysis should refer to corruption and transgressions of the code of conduct.

POINTS

DESCRIPTION

1

• The distinct impression is created that cases go unreported.
• No system to manage cases is in place. Nobody is responsible for managing misconduct. Records are poorly kept.
• No management reporting is done.
• < 20% of cases are dealt with within public service benchmark.

3

• System in place with explanatory manual. Person responsible for managing misconduct has been appointed. Good records are being kept.
• Reporting on management of cases is done
• 60% of cases are dealt with within public service benchmark.

5

• There is evidence that management doesn’t tolerate unethical conduct. Independent assessments of the level of corruption and opinions and qualifications in audit reports, correlates with the number of misconduct cases handled by the department.
• System in place with explanatory manual. Person responsible for managing misconduct has been appointed. Good records are being kept.
• Reporting on management of cases is done. Managers are held accountable for finalizing cases.
• 80% of cases are dealt with within public service benchmarks.

Context/ Constitutional Value

Efficient, economic and effective use of resources must be promoted.

Criteria/ Performance indicator

Expenditure is according to budget

Program objectives are achieved

Standards

• Expenditure is as budgeted
• Programmes are implemented as planned or changes to implementation are reasonably explained.

Underlying assumptions

Departments that

• Spend funds according to their budgets and>
• Achieve their objectives are better at using resources efficiently and effectively than those that over- or under-spend and are late in implementing their projects.

Policies governing this area

Treasury Regulations. Part 3: Planning and Budgeting

Public Service Regulations. Part III/B. Strategic Planning

Treasury Guidelines on preparing budget submissions. 2002

Treasury. Guide for the Preparation of Annual reports of departments for the year ended 31 March 2002.

Precidency. National Planning Framework.

The 2001 Estimates of National Expenditure (Red Book ) or similar document for the provinces and/or the strategic plan of the department.

The audited financial statements in the annual report or audit report.

Programme performance information from the annual report, management reports or performance information from departmental management information systems.

Researchers will have to determine verification procedures for assessing the credibility of performance reports provided by Departments. Use can be made of press reports and independent evaluation reports.

Format for presentation of quantitative data

• Budgetary analysis showing budgets and actual expenditure in bar chart form.
• Summary table showing departmental priorities, outputs, service delivery indicators and performance against the indicators.

Qualitative analysis

A narrative discussion should review the quality of the department’s service delivery indicators, its prioritisation, its management information system to collect data on the indicators and achievement against priority objectives. This should be no longer than 2 pages.

See note below on how to deal with multiple objectives/ indicators.

A narrative discussion should review the reasons for variation between expenditure and amounts voted.

1

• Indicators are not measurable, not understandable, are not measuring final outputs to the department’s clients, are not about the department’s own contribution towards outcomes
• Outputs and indicatorsdon’t reflect rigorous prioritisation.
• No management information system is in place to collect information on the service delivery indicators.
• Consequently no opinion can be expressed on service delivery performance of the department.
• Variation between expenditure and amounts voted is more than 10% for some programmes. Over (unauthorised) expenditure of the vote occurred.

2

• Less than half the indicators are measurable, understandable, are measuring final outputs to the department’s clients, and are about the department’s own contribution towards outcomes.
• Outputs and indicators doesn’t reflect rigorous prioritization.
• Performance information on some indicators is collected.
• In those instances performance was acceptable.
• Variation between expenditure and amounts voted is more than 10% for some programmes. Some programmes overexpended.

3

• Most indicators are measurable, understandable, are measuring final outputs to the department’s clients, and are about the department’s own contribution towards outcomes.
• The priority list is too long.
• A management information system to collect and report on information on the service delivery indicators is in place but disparate.
• 6 out of 10 priorities have been achieved.
• Variation between expenditure and amounts voted is more than 5% (but less than 10%) for some programmes. No overexpenditure occurred.

4

• Indicators are measurable, understandable, are measuring final outputs to the department’s clients, and are about the department’s own contribution towards outcomes
• Outputs and indicators reflect rigorous prioritisation.
• A management information system is in place to systematically collect and report on information on the service delivery indicators.
• 8 out of 10 priorities have been achieved.
• Variation between expenditure and amounts voted is more than 2% (but less than 5%) for some programmes. No overexpenditure occurred.

5

• Indicators are measurable, understandable, are measuring final outputs to the department’s clients, and are about the department’s own contribution towards outcomes
• Outputs and indicators reflect rigorous prioritisation.
• A management information system is in place to systematically collect and report on information on the service delivery indicators.
• All priorities have been achieved.
• Variation between expenditure and amounts voted is less than 2% for most programmes. No overexpenditure occurred.

Date

Step

Sept/ Oct

1. Departmental strategic planning. Departments develop medium term strategic priorities with measurable indicators and submit them to the Presidency by end October.

Mid November

2. Cluster-level integration. DG Clusters develop cluster-relevant priorities and indicators

Mid December

3. FOSAD produces a draft Medium Term Strategic Framework

January

4. Cabinet lekgotla. Cabinet approves the Medium Term Strategic Framework.

February

5. President’s State of the Nation address

February

6. Communication campaign. The aim will be to inform the South African public about the contents of the medium term priorities.

February/March

7. Consultation with provincial and local government

March/ April

8. Departments start their planning and budgeting so that they can submit their MTEFs to the Treasury by the end of June. (This means that the Medium Strategic Framework approved in January of 2002 imforms the budgeting process for 2003/04 – well in advance.)

Mid-financial year, financial year-end

9. Monitoring, evaluating and reporting

Context/ Constitutional Value

Public administration must be development-oriented

Criteria/Performance indicator

The Department effectively implements development programs that aim to alleviate poverty.

Definition of development projects

A narrow definition of development projects is adopted; otherwise we will collect information about a wide range of programmes, which will duplicate the information that we have collected under Performance Indicator 2. The following definition is suggested:

• Programmes targeted directly at the poor with the aim to enable them to provide their own livelihoods, including supplying their own basic needs.
• "Enable" means providing set-up grants, training, advice, and services like financial services, marketing and other business services.
• The definition does not include basic services, such as: (except if the services are produced by the beneficiaries themselves)

• Water
• Housing
• Income grants (such as old age pensions and unemployment insurance)
• Health services
• Education
• Training, where established industry is the direct beneficiary of the programme
• Electricity/ fuel
• Nutrition supplement/ feeding schemes
• Transport
• Infrastructure, such as roads, refuge disposal and sewage
• Job creation, where established industry is the direct beneficiary of the programme

Standards

Programmes are successful.

Beneficiaries participate in project design.

Good project management standards are maintained.

Organisational learning takes place.

Poverty alleviation projects and programs are integrated into local development plans.

Underlying assumptions

Departments that effectively implement development programs to alleviate poverty are more development oriented than those that do not.

All Departmental programs that meet the above definition will be reviewed. The relative performance of these programs will be assessed and measured in terms of their achievement of program objectives.

Data sources will include annual and other management reports, departmental management information systems, development programme manuals, project management manuals and interviews with development planners.

Verification will require assessment of independent evaluations.

Format for presentation of quantitative data

Programme performance data will be presented in summary tables.

Qualitative analysis

Qualitative analysis will be undertaken with regard to participation of beneficiaries in the design of projects, project management, organisational learning and alignment of programmes with local development projects.

1

• No information is available on the basis of which the success of the programme or projects can be assessed.
• No meaningful participation took place.
• Project management is totally dependent on individual project managers.
• Record keeping is chaotic.
• No meaningful organisational learning takes place.
• No integration with local development plans takes place.

2

• Aggregate information on programme level is not available. One out of ten projects were successful after three years.
• Beneficiaries didn’t participate in the design of projects but consultation took place.
• Project management is weak.
• Record keeping is haphazard.
• >No meaningful organisational learning takes place.
• No integration with local development plans takes place.

3

• Success is evaluated on both the project and programme levels. The programme was moderately successful. Three out of ten projects were successful after three years.
• Beneficiaries didn’t participate in the design of projects but consultation took place.
• A number of weak elements regarding project management standards were identified.
• Fairly good records are kept.
• The factors distinguishing successful projects were identified and discussed by project managers.
• No integration with local development plans takes place.

• Success is evaluated on both the project and programme levels. The programme was moderately successful. Three out of ten projects were successful after three years.
• Beneficiaries participated in the design of projects.
• Good project management standards were maintained.
• Project files should contain a complete set of standard project documentation.
• The factors distinguishing successful projects were identified and discussed by project managers.
• Local governments adjust their planning to fit provincially initiated programmes.

5

• Success is evaluated on both the project and programme levels.The programme is highly successful. Five out of ten projects were successful after three years.
• Beneficiaries participated in the design of projects.
• Good project management standards were maintained.
• Project files should contain a complete set of standard project documentation.
• The factors distinguishing successful projects were identified and discussed by project managers. Independent evaluation studies were commissioned.
• The programme is well-integrated with local development planning.

Context/ Constitutional Value

Services must be provided impartially, fairly, equitably and without bias

Performance indicator

The Department is making a concerted and coordinated effort to move towards compliance with the provisions of the Administrative Justice Act (AJA).

Standards

Prior notice of the nature and purpose of any administrative action is provided to any members of the public who will be affected by the action.

Opportunities are provided to make representation to people affected by administrative actions.

Clear statements are provided on any administrative decisions that get made.

A departmental policy for responding to requests for the provision of reasons for administrative decisions is in place

Requests for reasons for decisions are properly and reasonably processed.

The reasons provided for decisions are fair and reasonable.

Underlying assumptions

A Department that provides the reasons for its decisions and complies with the provisions of the Act is more likely to behave in a manner that is fair and impartial than one that does not.

The Act clearly requires certain actions from Departments in order to meet their legislative responsibilities.

Research in this area will focus on assessing the extent to which the Department is moving to meeting the requirements of the Act, even if it is not able to fully meet these requirements at this stage. The focus of the assessment of AJA implementation should be the degree to which the Department makes fair decisions.

Data sources will be in the form of interviews with officials in the Communication and Director General’s offices using a standard questionnaire.

Verification procedures still have to be determined but should include checking our assessment with an independent party such as an NGO or university research facility. The focus of the verification discussions should be on the fairness of the Department’s decisions making, rather than on its compliance with the Act.

Format for presentation of quantitative data

This data should be presented in the form of the data summary table described in item 14 below.

Further analysis should look at trends and patterns e.g. what kinds of requests receive faster or slower responses than others.

Qualitative analysis

The qualitative component should reflect on how much priority appears to be given to requests for information and how convincing and reasonable the responses provided seemed to the researcher.

The overall commitment to fairness should also be discussed using any evidence gleaned during the interactions.

SCORE

DESCRIPTION

1

There are few efforts made to comply with the provisions of the Administrative Justice Act and the Department is characterized by a reluctance to provide notice of or reasons for administrative actions.

2

Although some limited efforts have been made to become compliant with the provisions of the AJA, these are inadequate and require more commitment and direction. The Department does not have the necessary elements in place, such as training and policy, but has on occasion met certain requirements of the Act, although such compliance is sporadic and inconsistent.

3

There is a commitment to becoming compliant with the provisions of the Act and there has been some progress in this regard, for example through the provision of training and awareness initiatives, but much still remains to be done. Most important, the Department is moving towards providing notice and reasons for its administrative actions, and has records to support its claims in this regard.

4

A commitment to becoming compliant with the requirements of the Act is supported by activities and policies in the Department that will make this a reality. Limited training has been provided, or at the least awareness raising activities have been undertaken, and it is clear to the researcher that the spirit of the legislation has been incorporated into the operational and managerial style of the Department.

5

The Department is clearly committed to moving towards compliance with the provisions of the Administrative Justice Act and has made good progress in this regard. Training has been provided to staff on the legislation and responsibility for compliance has been allocated to an official. There is high awareness of the Act.

Context/ Constitutional Value

Peoples needs must be responded to and the public must be encouraged to participate in policy making

Performance indicator

The department facilitates public participation in policy making.

Standards

A policy on public participation is formally stated.

A system for soliciting public inputs on key matters is in use.

All policy inputs received from the public are acknowledged and formally considered.

A standard procedure or system for handling public suggestions and inputs is effectively implemented.

Underlying assumptions

Departments that have a policy and system for procuring public inputs to their policy-making processes are more responsive than those that do not and are more likely to integrate public opinion into their final policies.

Policy developed during the reporting period will be examined for evidence that public participation was used in its formulation and should also assess the extent to which public input was included in the final product.

Departmental policy and processes for generating, considering and responding to public submissions should also be reviewed.

Data sources will include copies of legislation passed during the reporting period, interviews with Departmental policy units and standard policy documents. The Cabinet Memorandum justifying legislation should also be drawn upon. Interviews with officials responsible for Parliamentary liaison and policy development should be interviewed.

If possible interviews with members of the public should be drawn upon: these may have been done for other projects such as the PSC’s Citizen’s Forums or for other consultative projects.

Verification procedures still to be determined.

Format for presentation of quantitative data

Quantitative data will include the provision of data tables listing policy proposals made during the reporting period and describing the number of public inputs received in relation to each.

Qualitative analysis

Qualitative data will involve reviewing departmental public participation procedures and considering these against a best practice standard.

The analysis should specifically consider why public inputs have not been included where this is the case.

Score

Description

1

There is little or no evidence of public participation in policy making. Policy and procedures to facilitate such participation are not in place.

2

A limited level of public participation in policy making during the reporting period can be detected. Policy and procedures to facilitate participation are not in place.

3

Around half the policies released during the reporting period has some evidence of public participation in its formulation. Policy promotes participation but procedures are not in place.

4

Public suggestions and inputs are included in most policies released during the reporting period. Systems and procedures are stated although not always followed.

5

All policies developed during the reporting period draw directly off public suggestions and inputs. All inputs received have been considered and responded to. Clear policy and procedures for public participation are in place.

Context/ Constitutional Value

Public administration must be accountable

Performance indicator

Adequate internal control is exerted over all departmental financial transactions.

Fraud prevention plans, based on thorough risk assessments, are in place and are implemented.

Standards

The Auditor General’s assessments of financial controls conclude that they are adequate and effective.

Fraud prevention plans are in place and are comprehensive and appropriate, and are implemented.

Key staff for ensuring implementation of fraud prevention plans, especially investigation of fraud, are in place and operational.

Fraud prevention plans are based on thorough risk assessment.

Underlying assumptions

Departments that implement financial controls and that prepare and implement fraud prevention plans are operating accountably.

he Auditor General’s assessments of Departmental financial controls are an adequate review of their efficacy.

Auditor General’s report

Fraud prevention plan

Risk assessment documentation

Number and experience of staff assigned to investigation of fraud.

Format for presentation of quantitative data

The number and experience of staff assigned to investigation of fraud will be presented in table format.

Qualitative analysis

This report will summarise and assess comments made in the Auditor General’s Reports.

The quality of the Fraud Prevention Plan will be reviewed for its compliance to good practice standards.

The risk assessment done by the department will be assessed against good practice standards.

Implementation of the fraud prevention plan will be considered against milestones described in the plan.

The department’s capacity to investigate fraud will be assessed.

1

• The Auditor-General has issued an adverse audit opinion or a disclaimer of opinion.
• No fraud prevention plan is in place.
• The department has no fraud investigation capacity.
• No risk assessment has been done.

2

• The Auditor-General has issued an adverse audit opinion or a disclaimer of opinion.
• A fraud prevention plan is in place but there is no commitment to implement the plan. The plan is academic.>
• The department has no fraud investigation capacity.
• No risk assessment has been done.

3

• The Auditor-General has issued a qualified audit opinion (except for matters highlighted, the financial statements are a fair presentation). Some lacks of internal control are mentioned under "Emphasis of Matter".
• A fraud prevention plan is in place and some progress has been made in implementing the plan.
• The department has some fraud investigation capacity.
• The fraud prevention plan is based on a risk assessment, though the risk assessment is somewhat superficial.

4

• The Auditor-General has issued an unqualified audit opinion. No lacks of internal control are mentioned under "Emphasis of Matter".
• A fraud prevention plan meeting all the standards for effective fraud control is in place and has been fully implemented. The plan is practical and suited to the department’s circumstances.
• The department has less than adequate fraud investigation capacity but has credible plans in place to boost capacity.
• The fraud prevention plan is based on a thorough risk assessment.

5

• The Auditor-General has issued an unqualified audit opinion. No lacks of internal control are mentioned under "Emphasis of Matter".
• A fraud prevention plan meeting all the standards for effective fraud control is in place and has been fully implemented. The plan is practical and suited to the department’s circumstances.
• The department has adequate fraud investigation capacity.
• The fraud prevention plan is based on a thorough risk assessment.

Context/ Constitutional Value

Transparency must be fostered by providing the public with timely, accessible and accurate information

Performance indicator

The departmental annual report meets the required standard and thereby facilitates transparency.

Standards

Standards have been clearly defined in an attachment. These are drawn from the National Treasury’s Guidelines and include the following:

• Standards with regard to the content of the annual report.
  
  
• The basic requirement that the annual report must be about performance against predetermined objectives (section 40(3)(a) of the Public Finance Management Act).
  
  
• Good reporting criteria such as comparing performance against targets or other benchmarks, understandability of the report, relevance of the information, verfiability of the information and explanation of performance indicators.

Underlying assumptions:

Departments that prepare their annual reports in accordance with National Treasury guidelines are committed to transparency and operate accordingly in other areas.

Policy/ Guides

National Treasury. Guide for the Preparation of Annual Reports of Departments for the year ended 31 March 2002.

Public Service Commission. Evaluation of Departments’ Annual Reports as an Accountability Mechanism. October 1999.

The standard published annual report is the data source.

No verification procedure required.

Format for presentation of quantitative data

Quantitative data will not be presented.

Qualitative analysis

A report assessing the quality of the Annual Report will be prepared according to a standard format. See Appendix 7.

1

The Annual Report barely complies with the Treasury guidelines and does not facilitate transparency.

2

A few of the necessary areas are addressed, but transparency is facilitated to a very limited extent.

3

The report covers many of the prescribed areas but also ignores key areas. Transparency in facilitated to a limited degree.

4

The Annual report complies with the guidelines and facilitates transparency but is inaccessible in certain limited areas.

5

The Annual Report complies with National Treasury guidelines, is well written in a concise and simple language, and facilitates transparency.

Context/ Constitutional Value

Good human resource management and career development practices, to maximise human potential, must be cultivated

Performance indicator

A.

Vacant posts are filled in a timely and effective manner.

B.

The Department complies with the provisions of the Skills Development Act.

Standards

A:

A recruitment policy complying with good practice standards and spelling out a detailed procedure, is in place.

Vacant posts are filled within a certain period.

There is a low vacancy rate unless recruitment has been frozen.

Regular management reporting on recruitment is done.

B:

A skills development plan, based on a thorough skills needs analysis, is in place

Activities planned for are implemented

The results achieved through skills development are monitored and recorded.

Underlying assumptions

A:

Effective recruitment policies and practices are a key indicator of good human resource management practice.

Departments that handle recruitment effectively and which fill their posts quickly and well are more likely to be maximising human potential than those that are not.

B:

Skills needs analyses are good instruments for assessing training needs and departments that draw upon them to prepare training strategies are working according to best practice.

Taking care to monitor performance against plan suggests that the department is committed to real human resource development.

A:

Recruitment, selection and appointment policy.

Human resources plan.

Establishment tables showing vacancy rate, staff turnover and time taken to fill posts (difference between date post became vacant and date of appointment in vacant post). Data source for this will be PERSAL establishment administration.

Management reports on recruitment, selection and appointment.

B:

Skills development plan.

Management reports.

Records of skills development activities.

Reports provided to the Department of Labor in terms of the Skills Development Act.

Assessments of the effectiveness of training commissioned by the department could also be utilized.

Format for presentation of quantitative data

A:

Data summary table showing establishment per occupational class and salary level, posts filled and vacant and number of staff who left the department in the period.

All posts falling vacant during the reporting period should be listed in a data summary table showing post, time taken from the date of the vacancy / post approval to the appointment of an officer/ employee.

A quantitative analysis of the time taken to fill posts can be done looking at levels as a correlating factor.

B:

A summary table listing what was achieved during the reporting period considered against what was planned; it should also reflect the reasons for any variance.

Qualitative analysis

A:

The quality of the recruitment, selection and appointment policy will be assessed against best practice.

B:

Consideration should be given to the extent to which the meeting of skills development targets had enabled the department to improve its service delivery.

1

• The vacancy rate and staff turnover is higher than the public service benchmark.
• < 20% of posts are filled within a timeframe equal to public service benchmark
• The department has no recruitment, selection and appointment policy in place.
• No management reporting is done.

2

• The vacancy rate and staff turnover is higher than the public service benchmark.
• < 60% of posts are filled within a timeframe equal to public service benchmark
• The department has a recruitment, selection and appointment policy in place but it does not fully meet good practice standards.
• No management reporting is done.

3

• The vacancy rate and staff turnover is the same as the public service benchmark.
• >60% of posts are filled within a timeframe equal to public service benchmark
• The department has a recruitment, selection and appointment policy that complies with good practice standards in place. The policy however does not contain creative recruitment strategies to meet the department’s special requirements.
• Only sporadic management reporting is done.

4

• The vacancy rate and staff turnover is lower than the public service benchmark.
• 70% of posts are filled within a timeframe equal to public service benchmark
• The department has a recruitment, selection and appointment policy that complies with good practice standards in place. The policy also provides for creative recruitment strategies to meet the department’s special requirements.
• Regular management reporting on recruitment is done and specific action is taken on the basis of the reports.

5

• The vacancy rate and staff turnover is lower than the public service benchmark.
• 80% of posts are filled within a timeframe equal to public service benchmark
• The department has a recruitment, selection and appointment policy that complies with good practice standards in place. The policy also provides for creative recruitment strategies to meet the department’s special requirements.
• Regular management reporting on recruitment is done and specific action is taken on the basis of the reports.

1

• No skills development plan is in place.

2

• Plans consisting of development activities to be undertaken are in place but the skills gap is not identified.
• Performance against plan was poor.

3

• Plans consisting of development activities to be undertaken are in place but the skills gap is not identified.
• Performance against plan was acceptable.

4

• A skills development plan meeting minimum requirements is in place.
• Performance against plan was acceptable.

5

• A skills development plan meeting minimum requirements is in place.
• Skills development targets are mostly all met. Impact of development on service delivery is assessed.li>

Context/ Constitutional Value

Public administration must be broadly representative of South African people, with employment and personnel management practices based on ability, objectivity fairness and the need to redress the imbalances of the past to achieve broad representation.

Criteria/Performance indicator

Departments are representative of the South African people and diversity management measures are implemented.

Standards

Employment equity plans are in place and reported upon.

Representivity targets are met.

Diversity management measures are implemented.

Underlying assumptions:

If the Department meets its diversity targets and demonstrates sound approaches to diversity management then it is likely to become representative in due course without compromising personnel management practices based on ability, objectivity and fairness.

The performance of the departments in meeting its diversity targets will be researched. A catalogue of diversity management efforts will also be prepared.

Reports submitted to the Department of Labour in terms of equity legislation for the reporting period will be a primary information source.

Format for presentation of quantitative data

See Appendix 9, Tables 1, 2 and 3. Report per occupational class and salary level the distribution for gender, racial group and people with disabilities against target.

Qualitative analysis

A qualitative assessment of diversity management will be done.

1

• Representivity targets are not met.
• No diversity management measures were implemented.
• No management reporting is done.

2

• Representivity targets are not met.
• No diversity management measures were implemented.
• Tables are provided in annual reports.

3

• The department is making good progress in achieving representivity targets.
• Some diversity management measures were implemented.
• Management reporting is done but diversity management measures not always followed through.

4

• The department is generally achieving representivity targets but some staff categories or levels, or some designated groups, are lagging behind.
• Some diversity management measures were implemented.
• Management reporting is done and appropriate action is taken.

5

• The department achieved representivity targets across various staff categories and levels.
• Innovative and practical diversity management measures are implemented.
• Management reporting is done and appropriate action is taken.

Background

Full name

Your name and position

Job Title

Date on which review was undertaken

From … to…

What period is covered by this annual report?

A high standard of professional ethics must be promoted and maintained.

Cases of misconduct are dealt with effectively and in good time.

Management System overview

Please describe the system used to manage and record cases of professional misconduct:

Is it computerized?

Is it centralised?

Who has access to it?

Who operates?

What post level is that person appointed at?

How secure is it?

Are cases of professional misconduct reflected in management reports?

Please comment overall on the quality of such reports.

How much detail is provided in these reports? (name of person charged and post details, offence details, process to date etc.)

How are these reports used? (To whom are they circulated, and what use do they make of them?)

Are you of the opinion that managers are aware of what cases are under way? Why do you think they are aware? How were they made aware?

How effective is the system?

Please express an opinion and justify your comments.

Does it ensure that cases move properly through the system?

What easy or low cost improvements could be made to the system?

Responses to and the finalisation of cases

Please state the total number of cases handled during the reporting period.

Comment on the main types of offences allegedly committed.

How does this relate to the core business of the Department concerned?

Please also state the number of cases concluded during the reporting period that arose before the period as well as the number of pending cases at the end of the reporting period.

What is the average length of time it takes to conclude a case of professional misconduct?

Should this be regarded as good or bad performance? Why?

Outcomes of cases

What proportion of cases led to a finding of Guilty? (%)

Should this be considered good or bad performance?

What proportion of cases was taken on appeal?

What were the main reasons for cases being taken on appeal?

How long on average did appeals take to be finalized?

Rating

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

CASE NUMBER

TYPE OF OFFENCE

NUMBER OF DAYS FROM DATE CASE REPORTED TO DATE FINALISED

(Guilty / Not Guilty)

(Yes / No)

COMMENTS

1

2

3

4

5

6

7

8

9

10

11>

12

13

14

15

16

17

18

19

Background

Full name

Your name and position

Job Title

Date on which review was undertaken

From … to…

What period is covered by this report?

Efficient, economic and effective use of resources must be promoted.

Program objectives are achieved

Expenditure is according to budget

Complete the table below.

Study the outputs and service delivery indicators in the 2001 red book (outputs, output measures/ indicators and targets in the 2002 red book), or the department’s strategic plan, and comment on whether the outputs and indicators –

• are measurable
• are understandable
• are measuring final outputs to the department’s clients
• are about the department’s own performance/ contribution towards outcomes

In the case of provincial administrations, use the similar book containing the province’s estimates of expenditure for the 2001/02 financial year.

• Public services are multi-dimensional. Many aspects of the service are important. Such aspects may be the resources invested in the service (inputs), the quantity and quality of the products and services (outputs) and the time it took to deliver the services, customer satisfaction with the service and the impact on communities and society (outcomes).
• Performance measurement or monitoring and evaluation require that the most critical dimensions of the services should be monitored by setting performance indicators.
• A performance indicator is an index, scale, count, or yardstick that gives a number or a statistic for a specific dimension of the service. It must be measurable.
• Output indicators may contain the quantity, quality and time dimensions.
• For each indicator an objective, target or a standard (service standard) should be set, that is the number that the indicator should ideally reach.

Obtain an authoritative document on the minister’s priorities for the department for financial year 2001/2002. List those in column 1. List the outputs and service delivery indicators associated with those priorities in columns 2 and 3.

Comment on prioritisation.

Get information on performance against the service delivery indicators from the department’s 2001/2002 annual report or from the department’s management information system and reproduce this in column 4. The performance information should use the same measuring scale as that implied by the indicator. For example: Not:

Indicator: 100% of nurses trained in advanced midwifery

Performance information: 153 nurses trained.

(I have no way of knowing whether 153 is 100%.)

Comment on the state of the department’s management information system to collect information on the service delivery indicators. Do this systematically for each indicator:

Are there procedures for collecting data with standard forms?

Are all the fields on the form exactly defined?

Are information on indicaors gleaned from transactional systems?

Is the system computerised?

Is it centralised?

Is responsibility to manage the system specifically assigned?

Is the system producing standard management reports at agreed frequency?

Comment on whether priority objectives were achieved. Do this systematically for each indicator.

Expenditure is according to budget

Obtain a copy of the department’s audit report for the 2001/02 financial year.

Turn to "Budget reconciliation and audit" (under the section "Emphasis of Matter". Calculate the amount not utilised as percentage of the total appropriation (amount appropriated, excluding roll-over PLUS improvement of conditions of service PLUS adjustment estimate).

Turn to the notes to the appropriation account. In one of the notes variations in excess of 2% between expenditure and amount voted are explained. Comment on the variations using the Auditor-General’s explanations as guide. Are variations related to performance issues or to factors beyond the department’s control?

Rating

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

1

2

3

4

5

Ministerial/ Departmental priorities>

Outputs

Service delivery indicators

Performance

Comments/ explanation

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

Background

Full name

Your name and position

Job Title

Date on which review was undertaken

From … to…

4. Reporting period

What period is covered by this report?

5. Constitutional Principle

Public Administration must be development-oriented.

6. Performance indicator

The Department effectively implements development programs that aim to alleviate poverty.

7. Success of the programme

Identify programmes meeting the definition of development projects.

Complete the table below. Evaluate the success of the programme by commenting on whether the targets have been achieved.

Comment on whether beneficiaries have participated in the design of the project.

9. Good project management standards

Evaluate whether project management adheres to good practice standards, using the checklist below. (Obtain a copy of the project management manual and check whether the manual provides for the items on the checklist.)

To check whether the project manual is implemented, check a random sample of 20 project files. Project files should contain a complete set of project documentation. Refer to point nr 15 on checklist. Comment.

Comment on whether the department knows why successful projects were a success. How did the department ensure that they learn from successful projects? Did they commission independent evaluation studies? Did they evaluate projects against critical success factors for development projects? This is essential for the success of future projects.

Describe the process that the department employs to ensure alignment between the targeting, budget allocation and objectives of the programme and local development plans. Comment.

Rating

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

Outcomes hierarchy

Outputs (examples)

Indicator

Target

Performance

Comments/ explanation

Incomes

Jobs created

Gender of beneficiaries

Sustainability

Skills and entrepreneurship developed

Infrastructure created

Grants paid

Hours of business support provided

Business plan implemented

Business plan approved

Business plan submitted

Feasibility study completed

1.1 Geographic area

1.2 Beneficiaries. Who are eligible for benefits?

√

2. Objectives of programme, types of projects

3. Funding

5. Standard project activities/ project cycle described

6. Responsibilities. Who manage projects?

How is the project --

8. proposed

9. assessed (What are the criteria for assessment of project?)

11. Obligations of beneficiaries

12. Monitoring of projects. At least the following should be tracked:>

• Completion of activities against schedule

• Expenditure against budget

• Performance against objectives

13. Disbursement of funds and financial controls

14. Evaluation of projects

15. Standard forms for the above, e.g. –

• Business plans
• Form for assessment of business plan
• Approval form
• Contracts
• Progress reports
• Evaluation reports

16. The support that will be provided to beneficiaries

Background

Full name

Your name and position

Job Title

Date on which review was undertaken

From … to…

What period is covered by this report?

Services must be provided impartially, fairly, equitably and without bias

The Department is making a concerted and coordinated effort to move towards compliance with the provisions of the Administrative Justice Act

Administrative decisions made by this Department

Please list the major administrative decisions made in this Department.

How do these decisions impact on the lives of ordinary South Africans?

Please explain the basis upon which the right to make and enforce the decision is based. List the enabling provisions for each major decision.

Are the business processes underlying each decision clearly mapped out – are there manuals and procedure documents available?

What information technology systems have been developed to support the work of the department in making their decisions?

How are decisions reached?

What records are kept of business processes?

Prior notice and opportunities to make representations

Does the Department provide affected people prior notice of its decisions?

How is this notice provided?

Is the provision of notice built into the business process or is it an exception made in certain cases?

>Are opportunities to make representations granted?

Is the public made aware of these opportunities?

How?

Provide examples of processes and opportunities for representation that are linked to decisions made during the reporting period.

Reponses to requests for reasons for decisions

Is there a system in place for handling requests for reasons for decisions?

Describe the system in detail, explaining how it works and who is involved in its administration.

If there is no formal system, explain how such requests are handled.

Review a small sample (up to 10 requests) of requests for the reasons for a specific decision.

Assess the outcome of each request and present this in a table form showing:

• The type of decision made and the query type,
• The length of time taken to respond,
• The nature of the final response,
• A short assessment of the fairness of the final decision.

Rating

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

Full name

Your name and position

Job Title

Date on which review was undertaken

From … to…

What period is covered by this report?

5.Constitutional Principle

People’s needs must be responded to and the public must be encouraged to participate in policy making.

6.Performance indicator

The Department facilitates public participation in policy making.

7.Existence and availability of policy

Is there a departmental policy on public participation in policy making?

What does it provide for? (Who should do what?)

How specific and measurable is the policy?

Is staff aware of the policy? Please provide reasons for your answer.

Are copies of the policy available? From where? To whom?

Comment on the quality of the policy and its scope.

Does it consider all types of situations?

Is it likely that situations will arise that are not covered by the policy?

Does it clearly explain what is to be achieved and how?

Are roles and responsibilities clearly defined?>

9.Recommended procedures

Does the Department have standard procedures for generating public comment and input on proposed policy?

What are these procedures?

10.Effectiveness of mechanisms

Does it appear as if the necessary procedures are followed in practice?

How effective do these procedures appear to be?

Use examples from the reporting period to justify your response.

11.System description

What system is used to receive public input on policy?

Describe this system in detail listing the steps and parties involved.

Are roles clearly defined and allocated or do they change depending on where the policy emanated from?

12.System effectiveness

Does the system described above operate effectively?

Draw on examples taken from the reporting period to justify your answer.

13.Inclusion of public comments

Describe in general terms the extent to which final policy draws upon public input.

Use examples from the reporting period.

In your opinion, was the policy generated enhanced as a result of public participation or not.

Provide detailed reasons for your answer.

14.Feedback mechanisms

Are people who contribute input or who commented on policy advised as to the outcome of their intervention?

How are they advised?

15.Score:

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

16.References

List all sources consulted in the preparation of this report.

This should include a list of documents, electronic sources and officials interviewed including the date and location of the interview.

Background

Full name

Your name and position

Job Title

3.Date of review

Date on which review was undertaken

From … to…

4.Reporting period

What period is covered by this report?

5.Constitutional Principle

Public administration must be accountable.

6.Performance indicator

Adequate internal control is exerted over all departmental financial transactions.

Fraud prevention plans, based on thorough risk assessments, are in place and are implemented.

Internal control

Obtain a copy of the department’s audit report for the 2001/02 financial year.

Check whether either the financial or compliance audit were qualified or not.

An unqualified audit opinion indicates that the auditor agrees with the information presented by management in the financial statements. However, an audit opinion may be qualified if the auditor considers that the financial statements do not present a fair view and the information may mislead the reader. These qualifications include issuing an adverse opinion (auditor not in agreement that it is a fair presentation), qualified report (except for matters highlighted, it is a fair presentation), or a disclaimer of opinion (when the scope of audit work required to form an opinion has been limited in some way). Lack of internal control is probably the major reason for qualifying audit opinions.

Check whether any lack of internal controls is mentioned in the audit report under the heading "Emphasis of Matter".

An audit report may also contain "emphasis of matter". This concept is used to draw attention to a matter that is considered important enough to be mentioned in the audit report, but does not prevent an unqualified audit opinion being expressed. Such matters are mostly lack of some internal control.

Fraud prevention plan

8.Fraud prevention plan

Obtain a copy of the department’s fraud prevention plan and evaluate it against the standards for effective fraud control strategies. Use the checklist below. Comment in two to three sentences on each standard. Is the plan practical and suited to the department’s circumstances?

9.Implementation of the plan

Check whether the tasks assigned to officers in the department (refer to standard 1 in the checklist below) have been completed. Comment.

Comment on whether key staff for implementing the plan, especially investigation of fraud, are in place. Complete the table below.

Risk assessment

10.Risk assessment

Obtain copies of the department’s risk assessment documentation.

Comment on whether --

• risks have been identified for all activities/ applications. (Activities/ applications in the Commission’s office are for example "doing service delivery projects", procurement, salary administration, or payments of creditors. It is important to note that the line function activities are also included.
• the seriousness of the risks have been assessed or whether risks have been prioritised. This is usually done by estimating both the likelyhood of the risk occurring and the impact of the risk.
• whether appropriate measures, including internal control measures, have been divised to address the risk.

Rating

11.Score:

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

Standard

√

Number of staff

Competence

Still learning the ropes

Adequate

Highly competent

Total

13

12

11

10

9

8

7

6

Lower

Total

Background

1.Name of Department

Full name

2.Name of reviewer

Your name and position

Job Title

3.Date of review

Date on which review was undertaken

From … to…

4.Title of Annual report

As on cover of the annual report.

Date especially important

Evaluate the quality of the annual report using the attached assessment questionnaire.

5.Reporting period

>What period is covered by this annual report?

6.Overview of report

How long is the Report?

What impression does it convey?

What superficial comments can be made?

Does it inspire confidence?

Are there typing or any other errors?

7.Information on the Ministry

Comment on the information provided on the Minister’s responsibilities and work programme?

Is enough information provided on:

1. Institutions falling under the Minister’s control?
2. Bills submitted during the period?
3. Official visits abroad (dates, purposes, costs, outcomes)?

8.Mission statement

Is this clearly stated?

Does it show why certain products and services are offered? Is the conceptual link clear?

9.Legislative mandate

1. Is the mandate explained?
2. Are the institutional arrangements identified?
3. Are roles and functions unpacked in the report?

10.Organisational structure

Is there a clearly depicted organisational structure showing branches and their functions?

Human resource overview

11.Establishment

Is there a detailed description of the establishment showing

1. >Approved and budgeted posts,
2. Numbers and grades of vacancies,
3. The objective behind each component?

12.Job evaluations

Is there information on job evaluations presented in the report? Is it comprehensive? (See 3.2 in the questionnaire)

13.Remuneration

Is remuneration clearly explained showing:

1. Total personnel, administrative and professional and speciall services costs
2. Deviations from CORE
3. Break downs by race, gender, disability, CORE>
4. Cost of the SMS
5. Costs of overtime and allowances?

14.Employment equity

Is enough information provided on

1. employees promoted;
2. Services terminated
3. Foreign appointees

Are these broken down by race, gender and disability?

15.Injury illness and death

Is there enough information provided on injury illness and death?

16. Collective agreements

Art collective agreements entered into by the Department adequately covered in the report?

17. Discharges due to ill health

Are discharges due to ill health adequately covered in the report?

18. Performance management

1. Does the report adequately cover rewards provided for performance, shown by race, gender, CORE and disability?
2. Is the number of employees in each category reflected in the report, properly broken down?
3. Is training expenditure against plans properly reflected?

19. Sick leave

Is sick leave taken by employees properly reflected, broken down by the necessary categories?

20. Disciplinary measures

Is information provided on disciplinary steps taken against employees, especially with regard to unauthorized, irregular or fruitless expenditure?

Programme Performance

21. Aims and objectives

Are the aims and objectives clearly stated and are these similar to those stated in the Red Book?

22. Programme performance overview (one by one)

Review each programme separately.

Head each comment: Programme One: etc. Comment as follows:

1. Is the aim of the programme clearly defined;

2. Are the achievements of the programme described at a result level rather than simply reporting activities? Are the achievements described in terms of the performance indicators listed in the Red Book?

3. Is it clear how this programme contributes to the strategic objectives of the Department?

4. Is there some analysis of how successful the programme was in meeting its intended targets and achieving its intended results? Are efficiency and effectiveness specifically considered?

5. Are the reasons for performance reviewed and considered?

23. Policy developments

Is there a short overview of strategic policy developments over the reporting period?

Financial information

24. Audit report >

1. Are the financial statements included in the Annual Report? Comment on these. This should include: balance sheet, income statement, cash flow statement and additional notes.
2. Is the Auditor General’s report on the Financial statements included? What impression do his comments convey?
3. Is the AG's report on the effectiveness of financial controls included in the Annual report? Please comment.

25. Additional notes in financial section >

Are the following issues addressed in the financial section of the annual report:

1. Transfer payments showing amounts transferred and to whom;
2. Contingent liabilities
3. Material losses
4. Unauthorised, irregular or fruitless expenditur
5. Foreign aid including amount, source and use
6. Pending applications for foreign aid.

Final comments on the Annual report as an accountability mechanism

26. Researchers comments

>Does the Annual Report facilitate transparency?

Background

1. Name of Department

Full name

2. Name of reviewer

Your name and position

Job Title

3. Date of review

Date on which review was undertaken

From … to…

4. Reporting period

What period is covered by this annual report?

5. Constitutional Principle

Good human resource management and career development practices, to maximize human potential, must be cultivated.

6. Performance indicator

a. Vacant posts are filled in a timely and effective manner.

b. The department complies with the provisions of the Skills Development Act.

Recruitment, selection and appointment

7. Staff statistics

Vacancies and turnover: Complete the table below.

Comment on the vacancy rate and turnover of staff. Is it high or low? Why? Was the size of the department’s establishment assessed recently? How?

Time to fill a post: Comment on how long it takes to fill a post. Is it long or short? Why?

Note: Consider adding a table on survival rate.

8. Policy

Obtain a copy of the department’s policy on recruitment, selection and appointment and also refer to the department’s delegations of authority. Evaluate the quality of the policy using the following standards:

1. The policy clearly describes the procedure to be followed.

2. Responsibilities are clearly assigned and commensurate authority delegated.

3. The policy ensures that:

• A proper assessment matches a candidate’s skills, knowledge and abilities with the work-related requirements of the job and the outcomes sought by the department , including representivity.
• The process is open, competitive and free of bias, unlawful discrimination, nepotism or patronage.
• Decisions are transparent and capable of review.

Does the policy/ human resources plan provide for creative recruitment strategies in circumstances where the department has difficulty in recruiting certain categories of personnel?

9. Monitoring

Is any management reporting done on recruitment, selection and appointment? Obtain copies of such reports. What is reported to whom? What action is taken on the basis of the reports? Comment.

Skills development

10. Skills development plan

Obtain a copy of the department’s skills development plan. Evaluate the plan against the following minimum requirements:

1. The plan must list the essential skills required to execute the activities of the department per post on the establishment.

2. It must then list the skills already possessed by staff, per post.

The difference between 1 and 2 is the department’s skills gap.

3. The plan must list the measures to acquire the skills to close the skills gap.

4. The measures must be prioritised and costed and a budget must be provided to execute the plan.

11. Performance against plan

Obtain a copy of a management report on skills development or other records on skills development activities. Evaluate the department’s performance against its own skills development plan. Prepare a summary table – Table 3 below. Has the impact of development activities on service delivery been assessed by the department?

Rating

12. Score:

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

a

b

c

d

e

f

g

h

i

j

k

Occupational class

Salary level

on 31.3.02

Filled on 31.3.02

Vacant on 31.3.02

Additional to establishmnt on 31.3.02

g/c%

Left dept period 1.4.01-31.3.02

(d+e+f)%

Permanent

Temporary

Out of adjustment

Total

a

b

c

d

e

f

g

h

Post

Date vacant period 1.4.2001 to 31.3.2002

Date advertised

Closing date

Date of interviews

Decision on appointment

Appointment date

g - b

Average

a

b

c

d

e

f

Skills category

Level of skill

Number of staff required with this category and level of skill

Number of staff already in possession of this skill

(plan)

(performance)

Total

Background

1. Name of Department

Full name

2. Name of reviewer

Your name and position

Job Title

3. Date of review

Date on which review was undertaken

From … to…

4. Reporting period

What period is covered by this report?

5. Constitutional Principle

Public administration must be broadly representative of South African people, with employment and personnel management practices based on ability, objectivity fairness and the need to redress the imbalances of the past to achieve broad representation.

6. Performance indicator

Departments are representative of the South African people and support for people from designated groups is provided.

Representivity

7. Numeric targets

omplete Tables 1, 2 and 3 below. Comment on whether the department is achieving representivity targets.

8. Diversity management

Obtain a copy of the department’s employment equity plan. Does the plan provide for measures, specifically targeted at designated groups, in the following areas? Are the measures provided for innovative and practical? Comment.

1. Specific objectives /targets to be achieved and activities to be undertaken and assignment of responsibility for achieving the objectives/ undertaking the activities.

2. Commitment from top management

3. Recruitment and selection

4. Skills development

5. Work environment (especially for people with disabilities)

6. Support, eg, mentoring

7. Awareness of employment equity and diversity issues

8. Work arrangements/ management styles that value diversity

9. Monitoring

What management reporting is done on representivity? Comment on the content, frequency, to whom the reports are addressed and the action that is taken on the basis of the reports.

Rating

10. Score:

What score do you award this department?

Please provide a detailed comment and explanation - use the Assessment Framework (Scaling section) as a prompt.

a

b

c

d

e

f

g

h

i

j

k

Number of staff

%

Occupational class

Salary level

African

Asian

Coloured

White

TOTAL

African

Asian

Coloured

White

TOTAL

TARGET

a

b

c

d

e

f

G

Number of staff

%

Occupational class

Salary level

Male

Female

Total

Male

Female

TOTAL

TARGET

a

b

c

d

e

Occupational class

Salary level>

Number of staff with disabilities

Total number of staff

%

TOTAL

TARGET

 Complaints & Compliments Form | Webmail | Disclaimer | Sitemap | Links