Rural Development Project
Transkrypt
Rural Development Project
Final Evaluation Study of the Rural Development Project Final Evaluation Study of the Rural Development Project Edt. Tomasz Majewski Warsaw, May 2005 AUTHOR: Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. FIRST IMPRESSION, WARSAW 2005 © Copyright by Foundation of Assistance Programmes for Agriculture Publication financed from the resources of the Rural Development Project All rights reserved; no part of this publication may be reproduced or transmitted in any form without the prior permission of the Foundation of Assistance Programmes for Agriculture. PUBLISHER Foundation of Assistance Programmes for Agriculture Rural Development Project Co-ordination Unit 00-930 Warsaw, ul. Wspólna 30 Poland tel. (+48....22) 623-19-01, 623-15-15, fax: 628-93-87 e-mail: [email protected], http://www.fapa.org.pl PRINTER Drukarnia cyfrowa DjaF, Kraków, ul. Misiołka 5, tel. (12) 294 45 27 Free of charge publication Final Evaluation Study of the Rural Development Project Final Report Contents I SUMMARY 11 1 Report Summary 12 1.1 Introduction 12 1.2 General Evaluation of RDP 12 1.3 Component A 13 1.4 Sub-Component B1 14 1.5 Sub-Component B2 16 1.6 Sub-Component B3 18 1.7 Component C 20 1.8 Evaluation of the Effect of External Financing from RDP on Investments Performed by Local Self-Governments 21 1.9 Analysis of the social and economic effects 23 1.10 Evaluation of Institutional, Organisational and Management Quality Related Solutions of the Project 24 2 SWOT Analysis II III IV 27 INTRODUCTION 29 1 Project Description 30 2 Context of Project Implementation 30 3 Objective of the Final Evaluation Study of the Project 31 METHODOLOGY 33 1 Research Model 34 2 Research Work and Data Collecting 2.1 Secondary Sources 2.2 Primary Sources 38 38 38 3 Analysis of Data 57 OUTCOMES 1 Component A 1.1 Top-down analysis 1.2 Bottom-up Analysis 1.3 Summary 61 63 64 82 89 2 Sub-Component B1 99 2.1 Top-down Analysis 100 2.2 Bottom-up Analysis 123 Annex 1 Supplementary Tables and Figures 146 Annex 2 Weaknesses and Threats Related to the Human Potential Activation according to SWOT Analysis based on Regional Development Strategies for the Years 20002004 (for Voivodships Participating in B1) 162 Annex 3 Description of Beneficiaries in a Breakdown by Services 165 3 Sub-Component B2 3.1 Top-down Analysis – Problems of Education in Rural Areas vs. Objectives and Outcomes of Sub-Component B2 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 167 168 5 Final Evaluation Study of the Rural Development Project Final Report 3.2 Bottom-up Analysis – Project Effects Evaluation 3.3 Summary 187 200 4 Sub-Component B3 4.1 Top-down Analysis 4.2 Bottom-up Analysis 4.3 Summary 209 210 238 248 5 Component C 5.1 Top-Down Analysis 5.2 Bottom – up analysis 5.3 Summary 5.4 Intergmina associations 259 260 292 315 326 6 Evaluation of the impact of external financing from the Rural Development Project resources on the investments realized by local governments 329 6.1 Identification of the subject of evaluation 330 6.2 Evaluation of the impact of external financing from the RDP resources on realization of investments 331 6.3 Summary and conclusions 348 7 Analysis of the social and economic effects 7.1 Analysis of the financial benefits for the state budget from the new jobs created 7.2 Direct investigation analysis of the costs and the social and economic benefits 7.3 Comparative (benchmarking) analysis of project effectiveness 7.4 Statistical analysis 7.5 Summary and conclusions 351 352 356 364 372 378 8 Evaluation of the institutional and organizational solutions and the quality of Project management 381 8.1 Introduction 382 8.2 The conception of Project management 383 8.3 Institutions and functions of the Rural Development Project in the light of program documents 386 8.4 The Poland Flood Emergency Project as a reference to RDP 389 8.5 Selected problems of the Rural Development Project management 392 8.6 Subcontractor selection procedures 413 8.7 Information systems in RDP 428 8.8 Opinions of the Project parties on its organization 432 8.9 Evaluation of the usefulness of RDP solutions for future projects 447 8.10 Risk management in RDP 461 8.11 Organization of RDP in the light of the NIK control 465 8.12 Summary 471 9 General appraisal of the RDP 477 9.1 Appraised Project 478 9.2 Presentation of Project Objectives 478 9.3 Analysis of the social and economic situation in Poland before launching the Project 479 9.4 Synthetic appraisal of Project objectives and goals 481 9.5 Assessment to what degree the Project fulfilled its objectives 486 V 6 FIGURES AND TABLES INDEXES 491 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Acronyms and institutions names used in the Report (in alphabetic order): General remark: for the purpose of the English version of the present report, the subject of evaluation is referred to as the Rural Development PROJECT, according to the terminology used by the World Bank. • ARMA – Agency for Reconstruction and Modernisation of Agriculture • ARMA RD – Agency for Reconstruction and Modernisation of Agriculture Regional Division • BDR – Regional Data Bank (Baza danych Regionalnych) • BGK – National Economy Bank (Bank Gospodarstwa Krajowego) • City and Gmina Administration Office (Urząd Miasta i Gminy) • CUI – Canadian Urban Institute • EPRD – Office for Economic Policy and Regional Development (Biuro Polityki Gospodarczej i Rozwoju Regionalnego) • EU – European Union • FAPA – Foundation of Assistance Programmes for Agriculture (Fundacja Programów Pomocy dla Rolnictwa) • FDPA – Foundation for the Development of Polish Agriculture (Fundacja na Rzecz Rozwoju Polskiego Rolnictwa) • FEP – Flood Emergency Project (Program Usuwania Skutków Powodzi) • FDL – Local Donations Fund • FLDD – Foundation for Local Democracy Development (Fundacja Rozwoju Demokracji Lokalnej) • RDF – Rural Development Foundation (Fundacja Wspomagania Wsi) • Gmina Administration Office (Urząd Gminy, UG) – the basic level of self-government • Main Statistical Office (Główny Urząd Statystyczny, GUS) • Institute for Market Economy Studies (Instytut Badań nad Gospodarką Rynkową, IBnGR ) • IDP – Institutional Development Program • IOPRD – Integrated Operational Plan for Regional Development (Zintegrowany Program Operacyjny Rozwoju Regionalnego) • MARD – Ministry of Agriculture and Rural Development • MIAPA – Ministry of Interior and Public Administration Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 7 Final Evaluation Study of the Rural Development Project Final Report 8 • MIS – Management Information System • Marszalek Office (Urząd Marszałkowski, MO) – the voivodship self-government is a regional self-governing community and the largest unit of the territorial division of the state. Then voivodship Marszlek is the chairman of the voivodship board. The Marszalek Office manages the daily affairs of the voivodship, represents it and provides assistance to the Voivodship Board. • Middle school - (Polish: gimnazjum) the level of education between the primary and the scondary school, often referred to as "gymnasium". The term "middle school" has been introduced to avoid confusion with the English word "gymnasium", or "gym" meaning the school sports hall. • MoEL – Ministry of Economy and Labour (formerly: MoLSP – Ministry of Labour and Social Policy, MoELSP – Ministry of Economy, Labour and Social Policy – these acronyms are interchangeable) • MoF – Ministry of Finance • MoNES – Ministry of National Education and Sports • Małopolska Szkoła Administracji Publicznej, MSPA (Małopolska School of Public Administration) • Superior Chamber of Inspection (Najwyższa Izba Kontroli, NIK) • NSC – National Steering Committee • NSC – National Steering Committee • PCU – Project Co-ordination Unit • PLO – Powiat (District) Labour Office (Powiatowy Urząd Pracy, PUW) • PNBA – Polish Network if Business Advisers (Polska Sieć Doradców Biznesu) • Powiat Starosta Office (Urząd Starostwa Powiatowego) – powiat is a local selfgoverning community and covers a whole area of adjacent gmina, but does not supervise their activities. Powiat is headed by Starosta (Chairman of the Board). • PT – Project Team • RDP – Rural Development Project • RP – Redeployment Program • SOP – Sector-related Operational Program (Sektorowy Program Operacyjny) • TSU – Territorial Self-government Unit • Voivodship Administration Office (Urząd Wojewódzki, UW) – Central Government regional representation • VLO – Voivodship (Regional) Labour Office (Wojewódzki Urząd Pracy, WUP) Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report • VPIU – Voivodship Project Implementation Unit • WB – World Bank Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 9 Final Evaluation Study of the Rural Development Project Final Report I Summary Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 11 Final Evaluation Study of the Rural Development Project Final Report 1 Report Summary 1.1 Introduction RDP was accepted for implementation with the following overall objectives in view: • increased non-farm employment in rural areas, • support for the process of strengthening local and regional self-governments and support for regional development, • support for the process of building the institutional capacity necessary to obtain EU pre-accession and structural funds. The plan was to implement a wide range of activities. With regard to their variety, they were divided into three groups, called Project Components. The following Components were designed: A – Micro-credit, B – Human Capital Development, C – Rural Infrastructure. Additionally, three Sub-Components were distinguished within the frames of Component B: B1 – Labour Redeployment, B2 – Education, B3 – Institution Capacity Building in Local and Regional Administration. 1.2 General Evaluation of RDP RDP was a very complex development undertaking financed from public funds and implemented in a difficult social and economic environment. It preceded Poland’s accession to the European Union, therefore it had an important role to play in preparation of infrastructural, institutional, social and economic bases for integration of our country with EU. The selection of objectives and activities accepted for RDP was correct and responded to the most important needs in the field of rural areas development in Poland. The Project was implemented in very difficult macro-economic conditions: slowed pace of GNP growth and rapidly growing unemployment the rate of which exceeded 18% in this period. These circumstances made it impossible to achieve all RDP objectives, especially to increase the non-farm employment in rural areas1. RDP was the first such an extensive program supporting the development of rural areas in Poland in the period of post-communist transformation. Programs implemented so far had been of a sectoral nature and most of them had been oriented towards agriculture problems first of all. The main focus of RDP was on the most important needs related to development of the given territory (rural areas in particular territorial units). In this respect, this was a pioneering project and it paved the way for similar EU-financed programs. Livening up and activating self-government authorities and business entities and subjects using particular activities encompassed by RDP Components was an important effect of the Project. The public and financial mobilisation stimulated by the Project is of a positive and long-term significance for rural areas development. 1 We are stressing that the objective related to creating off-farm jobs was planned to be achieved within some 5 years after completion of the investments, i.e. for the years 2006-2009, as the investments were effected in the years 2001-2004. 12 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Should one apply the “school” grading (1-5) to the overall evaluation of RDP, the final mark is above 4. According to the terminology used by the World Bank, this mark is between “satisfactory” and “highly satisfactory”. The evaluation team did not observe any significant failure in Project implementation or variances with the assumptions and objectives. The Project enjoyed a good reputation in local environments proved by our questionnaire surveys and most of the procedures were found “friendly”. The respondents were also of a high opinion about competence and approach to the beneficiaries shown by persons responsible for implementation of particular elements of the Project. The funds provided by the Loan Provider (World Bank) were used in 100% nearly and the efficiency of expenditure can be considered high. Quantitative indicators were exceeded in most Components of the Project and this fact shows that costs of outputs produced as a result of the Project were lower than assumed. Thus, the loan contract has been executed entirely. 1.3 Component A 1.3.1 Key conclusions • Before the year 2000 the situation of regions covered by Component A – especially in Warmińsko-Mazurskie and Podkarpackie voivodhips – had been worse than in other voivodships. The unemployment-related problems identified in the selected rural areas had been of a deepening and long-lasting nature. In our opinion, the intervention in the form of Component A was justifiable. • The Component objective related to promotion of micro-businesses enabling the creation of off-farm jobs identified voivodhips’ problems and needs correctly. Component activities were selected well, especially that the approach to individuals who had already been running their own business differed from the approach to those who were just beginning. It seems valuable that financial instruments were supplemented with training and advisory assistance. When filling the questionnaires, the respondents gave very high marks to the utility of financing in the form of microcredits and grants combined with advisory assistance (74% of very good grades). The relevance of Component activities is assessed as satisfactory. • The impact of the Component at the regional and voivodship level was limited (the attendance was below 1% of potential beneficiaries) due to the fact that the funds were relatively modest. The structure of beneficiaries who participated in Component A should be assessed positively. The major group of beneficiaries consisted of persons beginning their own business activity (65%) and persons based in rural areas (64%). Women constituted a significant part of Component participants (43%). On the other hand, the attempt to reach one of the potential groups of beneficiaries – school graduates – was unsuccessful. Not many young people participated in the Component in general. The attendance is assessed as satisfactory. • Quantitative outcomes of the Component were beyond the expected performance, despite the fact that the time of implementation had been reduced in practice. 2941 new job placements were created as a result of Component activities. 4655 microcredits and 2944 grants were provided to the beneficiaries. The effectiveness of the Component is assessed as highly satisfactory. • Project targeting is assessed as satisfactory. The Component objective was consistent with the Government’s efforts to reduce the unemployment in rural areas and to diversify the rural population’s sources of income through development of non- Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 13 Final Evaluation Study of the Rural Development Project Final Report farm employment. Component A fit in the State policy in the field of rural areas manifested in – inter alia – the Cohesive Structural Policy for the Development of Rural Areas, the Treaty for the Agriculture and Rural Areas and the Mid-Term Strategy for Agriculture and Rural Areas Development. • The comparison of benefits generated by the Component with its costs is highly satisfactory – we are estimating the period of this investment return at a year and a half. We are assessing the cost of creating one job placement as relatively low. To conclude it, the efficiency of the Component is assessed as highly satisfactory. 1.3.2 Advantages • The mechanism of granting credits turned out to be very effective, as the assumed quantitative outcomes have been exceeded despite the fact that the period of Component implementation was much shorter than planned. • It seems that the fact of selecting good loan advisers who often approached potential beneficiaries in a very active manner was one of the reasons of high effectiveness here. It is worth mentioning that the beneficiaries expressed very positive opinions about the work of loan advisers from foundations implementing Component A (63% of very good and 28% of good marks). 1.3.3 Disadvantages • In the first two years there were many problems with launching the process of microcredits related to selection of service providers. After the process was launched, financial means were disbursed to beneficiaries with serious delays. In practice, Component A starting period took more than two years. In our opinion, the most important problems were eliminated no sooner than in the second half of Component implementation and effective mechanisms of micro-credits distribution were set up. • Little use was made of the potential synergy with Sub-Component B1. 433 persons participating in both Components – A and B1 – have been identified. 1.4 Sub-Component B1 1.4.1 14 Key conclusions • The analysis of objective secondary data has shown that before the year 2000 (i.e. before Sub-Component B1 was launched), the situation of regions to be covered by it was worse than in the remaining voivodships. What is more, the labour-market related problems were of a long-lasting nature and market mechanisms did not lead to any solution. Thus, the intervention in the form of a program like Sub-Component B1, oriented towards labour redeployment has been assessed as justifiable. • The comparison of Sub-Component B1 with problems identified in regional strategies showed a far-reaching convergence indicating that Sub-Component B1 identified these voivodships’ real problems and responded to them correctly. This opinion was also confirmed by a significant part of the respondents, although there was a group of them – aged people with university degrees and low income – who indicated program irrelevance. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report • The analysis of situation in the voivodships covered by Sub-Component B1 in comparison with the remaining regions after project implementation showed that the situation of regions covered by Sub-Component B1 relatively improved in some aspects. One should stress however that the respondents’ general opinion seems to indicate that the scale of intervention was too small when compared with the needs, to be able to speak of Sub-Component B1 ability to solve the problems, but this was a step in the right direction. • As far as the scale of participation is concerned, it has been estimated that about 15% of persons eligible for the project participated in it on the average. The respondents, beside reporting the fact that there were not enough places to accommodate their participation, indicated also other problems that set limits to their participation, such as insufficient information, disbelief that the program could change their situation on the labour market, program inadequacy to their needs and costs of travelling. • The effectiveness of Sub-Component B1 planned outputs achievement has been evaluated on the basis of objective indicators and based on respondents’ opinions as well. Although the effectiveness varied from service to service, it exceeded 80% of the assumed performance on the average. Good effectiveness of the project is also confirmed by outcomes of interviews and questionnaire surveys. • Although the immediate general outcome of the project (i.e. decrease of unemployment in the selected voivodships) was not evident (due to the scale of the Redeployment Program), respondents’ opinions indicated an important additional impact of Sub-Component B1 in the form of increased self-confidence and stimulated activeness of those persons experiencing problems with employment who had not been participating in any similar programs in their majority. • It is hard to identify the sustainability, as not much enough time has passed since the project closing. The respondents’ opinions concerning the sustainability of impact varied. The beneficiaries themselves believed in it more than the officers who were responsible for the project and who had a more comprehensive knowledge of the labour market. • As far as the cost/benefit proportion is concerned, it should be stressed that some of the costs might have been distributed among the beneficiaries who declared in their questionnaires that they could pay a part of the costs of services they were provided (this would be possible with the adequately defined criteria of participation). On the other hand, one should emphasise those unintended – mostly positive. 1.4.2 Advantages • The very good construction and planning of the Redeployment Program should be considered as its greatest strength – particularly, the idea to integrate the Program with other elements of policies of a multi-sectoral nature, i.e. with loans, infrastructure development, education. No approach like this – i.e. orientation towards synergy – to the complex matter of the rural labour market problems had existed in previous programs of this type. This approach enforced co-operation between various ministries and allowed for positive external impact. • Moreover, introduction of the element of survey aimed at identification of labour markets’ needs and capacity before RP implementation was an advantage. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 15 Final Evaluation Study of the Rural Development Project Final Report 1.4.3 Disadvantages • Lack of co-ordination with Component A (micro-loans) which started with a serious delay and this fact affected business assistance centres and business incubators activity negatively. The RDP construction allowed for potential synergy between Component A and Sub-Component B1. It had been assumed that persons trained under Sub-Component B1 would have wished to launch their own businesses, using micro-loans and grants. As a result of the circumstances referred to, the effects of synergy were much weaker than those potentially possible. • Serious delays in disbursement of funds for service providers resulted in a reduced participation of smaller companies having no sufficient financial potential of their own. • Although Sub-Component B1 orientation towards disadvantaged inhabitants of rural areas achieved by the means of skilful formulation of the eligibility criteria was one of the greatest advantages of the project (when compared with other programs, such as PHARE), the reality of project implementation did not fully follow the guidelines. This means that the disadvantaged persons, owners of very small farms and school graduates from rural areas constituted a small percentage of Sub-Component B1 beneficiaries. 1.5 Sub-Component B2 1.5.1 16 Key conclusions • The decision to implement Sub-Component B2 under RDP should be considered as justifiable. The fact of planning educational Sub-Component B2 within RDP had been deeply justified with education-related needs of rural areas. Gminas’ low budgets do not enable them to finance renovation works and supply of didactic aids needed in schools A vast majority of gminas would not make so many necessary refurbishment and modernisation works, would not buy so many didactic aids and/or software without the financial support. Fewer teachers would participate in training – due to financial reasons and limited access to centres running training courses for teachers. • The objectives and the tasks of Sub-Component B2 should be considered as relevant to the needs and targeting the States’ education policy well. The objectives and the activities of the Project fit in the program of enhancing the quality of education in rural areas and the guidelines of the educational policy of the State comprehensively – it covered such aspects as implementation of the education management reform, curricula reform, professional promotion of teachers and school network consolidation (including organisation of pupils transportation). • The expected outcomes (quantitative indicators) were clearly exceeded in most cases. This applies in particular to rehabilitation activities. On the other hand, the achieved indicators were lower than planned in such tasks as didactic training for teachers, IT training and Scholaris Internet portal. This resulted mainly from the fact that rehabilitation-related needs prevailed and the procedures were so protracted that teachers’ needs changed in the meantime. The very idea to create an educational portal should be assessed high – it is becoming increasingly popular. Considering the outcomes that have been achieved and their long-term impact, the activities of the Sub-Component should be considered useful. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report • The level of attendance can be considered satisfactory, although the needs of potential beneficiaries are much greater in the field of rehabilitation and investment activities. The participation was regulated by the eligibility criteria, by gminas’ financial potential (own contribution) and by the Sub-Component funds. The eligibility criteria made it difficult for small schools to apply, although these needed support very much. The demand for financial support varied from task to task – it was satisfied almost entirely in case of didactic training and IT training, as it turned out to be lower than expected. Serious reductions of applications took place in the group of rehabilitation tasks only, although the demand was immense here. • The implementation of Sub-Component B2 tasks gives a chance for a long-term impact. Refurbished and equipped buildings and premises should serve students for many years. The idea to include a new task – equipping pre-school education classrooms – should be appreciated, although there was less interest in it than in case of previous tasks. But well equipped pre-school education classrooms for youngest inhabitants of small farming gminas should provide a better educational start and further school career to children in rural areas, where very few children go the kindergartens. Training and supply of didactic materials and educational software should translate into increased level of education in rural areas. The sustainability of investment impact should be estimated – depending on the scope of works – at a few years or at very long years. The sustainability of the effects of training and purchasing should be strengthened by the means of further activities, continuous refreshing of the didactic base and continuous development of teachers’ competence. The one-off support provided within a scope which was limited in terms of quantities and territory can not ensure any sustainable long-term impact. • The rules of economy were observed. Refurbishing and investment tasks were performed at the cost below the plan and as a result of this schools from more distant places in the ranking could be included in the Project. Costs of training and prices of didactic materials and software did not differ from those quoted on the market. It seems that Sub-Component activities were organised in a manner enabling as many beneficiaries as possible to be included. Besides, gminas made efforts to minimise their own Project-relate costs. The assumed cost level was exceeded in such areas, where better outcomes were achieved to the advantage of beneficiaries. 1.5.2 Advantages • Inclusion of education related tasks in the Project. • Comprehensive approach to education-related needs. • Simple application procedures. • Very good background provided to beneficiaries (though training) in the field of applying for funds. • Allocation of additional funds for rehabilitation tasks in the further phase of the Project. • Giving up the rule of reimbursement. • High quality of refurbishing works. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 17 Final Evaluation Study of the Rural Development Project Final Report • High quality of didactic training and cascade effect. 1.5.3 Disadvantages • Small schools eliminated in practice (although not all of them will be liquidated) in consequence of the number of students in school” criterion • Too long time of waiting for financial support (particularly in case of refurbishment and investment works). • Too long time of waiting for training. • Too short time for refurbishing works at the II stage, after re-location of funds. • Central bidding procedure for companies offering didactic materials and software. 1.6 Sub-Component B3 1.6.1 18 Key conclusions • RDP was necessary in the spectrum of Sub-Component B3 in the period when it was being implemented, as it prepared the Polish self-government for joining EU structures. The Sub-Component objectives complied with the State policy objectives outlined in such strategic documents as the National Strategy for Regional Development, the National Strategy for Employment Growth and Human Resources Development and the Cohesive Structural Policy for Agriculture and Rural Areas Development. Thus, one may state that the relevance of Sub-Component B3 was high, as it identified the real problems and needs of the target group correctly and responded to them. • The effectiveness of Sub-Component B3 in general should be considered very high at the level of IDP implementation in gminas. Nevertheless, the efforts aimed at implementation of tools proposed within B3 to transform the model of TSU management considerably were not entirely successful. The bureaucratic model of management still remains very common. The strengthening of transformations through the legislation process should increase of effectiveness of IDP significantly. • On the other hand, through the implementation of the All-Poland Training Program, Sub-Component B3 contributed to the Polish self-governments’ preparedness for absorption of pre-accession and structural funds. The number of applications submitted to SAPARD and IOPRD, as well as the utility of training provided under the Program declared by the participants give evidence to this conclusion. • Similarly as in case of effectiveness, the evaluation of Sub-Component B3 utility can not be clear-cut. The utility of All-Poland Training Program deserves a positive assessment definitely – in this case, the participants themselves stressed that they were able to use the acquired knowledge and skills in their work on project applications. The utility of tools approved by gminas during IDP implementation should be assessed highly too. On the other hand, the fact that no activities are taken up to continue dissemination of tools offered to TSU within the frames of IDP raises some concerns. Although many beneficiaries declared that they did not implement further innovations, as these were inadequate to their needs, one may presume that the trend towards modern management will make the beneficiaries appreciate the Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report solutions they were offered and the changes in the culture of management in the nearest future. Nevertheless, these transformations – as any cultural transformation require time, as they involve changes of awareness and of the style of work. The fact of using the corruption counteracting tools and participating in “Transparent Poland” competition can be given as an example of this trend. • The sustainability of Sub-Component effects will also be high in case of the Training Program, as it will be transferred to the achievements of projects implemented directly or indirectly as a result of participation in training. • In case of IDP impact sustainability analysis, one should state that some of the implemented tools have become routine procedures in TSU. Similarly as in case of the Training Program outcomes, they will contribute effectively to TSU’s success in obtaining pre-accession and structural funds. But in most cases, the activities did not add to creation of new jobs or entrepreneurship development in gminas, or to any improvement of services quality in gminas. Thus, the sustainability of the Program impact will be connected with the effects of funds absorption, while no immediate qualitative change if institution’s work can be expected. To make the effects of IDP and of IDP dissemination sustainable, transformation in TSUs’ organisational culture should be continued and more legislative support should be provided. • It is difficult to sum up the evaluation of Sub-Component B3 efficiency, as, due to the high innovativeness of its activities for the self-government sector, no comparative data are available. Should one assess the efficiency of solutions resulting from IDP, the evaluation should be made on a long-term basis. 1.6.2 Advantages • Mass participation of TSUs in the Training Program and suitable time of its implementation. • Making gminas more sensitive to corruption problems. • Increased efficiency of gminas in obtaining funds – pre-accession and structural first of all. • Commencing the process of institutional transformations in the self-government administration adapting them to functioning in EU – efforts made to transform the TSUs’ organisational culture. 1.6.3 Disadvantages • The good practice database potential has not been used as it could have been. • MIAPA did not take up an active participation IDP dissemination activities after SubComponent B3 of RDP was closed. • Public administration’s lack of organisational capacity for unaided implementation of the transformation program. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 19 Final Evaluation Study of the Rural Development Project Final Report 1.7 Component C 1.7.1 20 Key conclusions • The analysis of the rural infrastructure development shows that before the year 2000 the main problems of rural areas in Poland included: a very low level of infrastructural equipment (accepting EU rural areas and Poland’s urban areas as reference points), huge disproportions in this field between regions and voivodships and on the other hand – within their borders, and dissonances between development of particular infrastructural systems in local arrangements. The confrontation of the scale of demand for infrastructural investments with the limited financial potential of local selfgovernments justified the financial support for infrastructure under RDP. In our opinion, the strategy of covering all 16 voivodships accepted for Component C was justifiable, first of all due to the possibility of selecting the most efficient investment projects in Poland for co-financing. • The comparison of Component C objectives with the problems identified in strategies for rural areas development reveals a far reaching convergence which shows that Component C identified problems of rural population correctly and responded to them. The role and the significance of infrastructure in local development is proved by the fact that representatives of local authorities, businessmen and inhabitants agree that the lack of infrastructure is the main barrier to the social and economic development of gmina. The utility of infrastructural developments made under the Project was assessed as very high. This gives a basis for a positive evaluation of relevance of objectives accepted for Component C to beneficiaries’ needs. • The analysis of situation in gminas covered by Component C shows that in case of some activities (water supply and sewage collection systems), the increase in the number of infrastructural devices in the group of Project gminas was higher than effects achieved on a national scale in the years when RDP was implemented (considering proportions of participation to the effect achieved). Moreover, the difference in dynamics of changes in the increase of the number of infrastructural devices between Project and non-Project gminas was very clear to the advantage of the first group (in case of water supply terminals this was twice as high and in case of sewage collection terminals – by 30% higher). The Component activities contributed to a 10% reduction of the difference between the number of water supply terminals and the number of sewage collection terminals on the national scale and a 30% reduction in the group of gminas covered by RDP. Component C should be assessed as very useful in terms of improvement of rural infrastructure in gminas participating in RDP. • Taking the number of gminas applying for funds from RDP and the number of gminas these funds were granted to as a basis, the participation in the Project should be assessed as high. In practice, 30% of potential beneficiaries participated in investment activities of the Component, while training activities were provided to all beneficiaries. • In general, the outcomes planned for the whole Component C were achieved. The outcomes varied in particular activities, but the situation when more was achieved than planned was more frequent. Moreover, the beneficiaries gave very good marks for effectiveness of the Component. A vast majority of them expressed an opinion that the assumed objectives had been achieved. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report • Immediate outcomes of Component C have been achieved, the infrastructural equipment of rural areas has improved – this is confirmed by objective indicators and by opinions expressed by representatives of various social groups as well. Infrastructural investments contributed to improvement of the rural population’s living standards (including health conditions) and of the conditions of running business activities. The investments resulted in a reduction of some companies’ financial burden, in equipment upgrading (this happened in every fifth of the companies covered by the survey), in increased scale of production (16%), increased range of activity (14%) and increased employment (13%). Due to the Component, 28% of the companies covered by the survey survived the “hard times”. As a result of investments the investment attractiveness of gminas has grown, this thesis being confirmed by the increased interest in purchasing building and recreation plots. The infrastructural investments did not contribute to the increase of new employment opportunities to such a degree as had been expected. The Project assumptions concerning the range of infrastructural investments’ effect on the living standards and facilitation of running business activities turned out correct. • The analyses and studies of benefits resulting from the infrastructural investments have shown that it is a very difficult task to covert them into monies. Nevertheless, they enable one to state that the impact of these investments on economy and of working and living conditions of rural population is positive. This also what the beneficiaries feel. 1.7.2 Advantages • Very good co-operation of VPIUs with local self-governments. The project personnel very competent and helpful in practice, at the stage of application and project implementation. Genuine and skilful (immediate) help, openness to co-operation, friendly approach to beneficiaries. • High utility of training provided within the frames of Component C, first of all in its practical dimension and in terms of utility for projection of infrastructural investments. • The Component was definitely easier and more “friendly” than other programs being implemented now (e.g. SAPARD), the “level of bureaucracy” lower than in other programs. 1.7.3 Disadvantages • Too long time of waiting for money and too long period between the application submission and its acceptance and approval for implementation. • Excessive reporting – the beneficiaries questioned also the need to submit the required reports, especially the frequency and the requirement to report even when nothing had changed. 1.8 Evaluation of the Effect of External Financing from RDP on Investments Performed by Local Self-Governments To sum up the results of investments performed by local self-governments, one may state that: • The time limit planned for investments was observed. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 21 Final Evaluation Study of the Rural Development Project Final Report • The costs incurred under Component C were lower than the costs envisaged at the stage of applying. This results mainly from differences between the value calculated before the bidding and the value of a contract signed with the contractor selected as a result of the bidding procedure (investment overestimation) and also from the fact that the general scale of investments was reduced, as beneficiaries withdrew from some of investment tasks planned in their applications. The value of the contract increased as a result of additional works, the necessity of which emerged during investment realisation. The reduction of the scope of works can be considered insignificant on the whole Project scale. • Much more funds were spent within the frames of Sub-Component B2 than it had been planned in PAD before the investment was launched, as in the year 2003 the Program was extended over all voivodships (while 7 had been planned) due to reallocation of funds from the Project reserve to this Component. Therefore, the material status of the investment was higher than assumed initially, in proportion to funds allocated for this purpose. • The protracted process of transferring funds to gminas’ accounts should be the crucial element disturbing Project implementation in its initial period. Initially, the period between acceptance of an application for funds and funds receipt by gmina was as long as sic months in some cases. This might cause conflicts with work contractors waiting for their payments. In some cases, gminas contracted commercial loans to settle financial liabilities to contractors and this caused additional burden on gmina budgets. Later on, this inconvenience was eliminated and the time of waiting for money was reduced significantly. It was advantageous for the beneficiaries that investments were financed in tranches, as gminas did not need to engage funds amounting to the entire co-financing value. Such a high own contribution might have been a serious difficulty and in some cases – even a barrier to taking up the investment task. • The costs of facilities built under the Project remain similar to costs of building such facilities all over Poland. • Unit cost indicators for building developments are similar to indicators achieved in other assistance programs. • Due to the dispersed and local nature of particular undertakings and their relatively small size, the reduction of unit outlays is not perceptible (the effect of scale). Data analysed during the final evaluation of infrastructural investments performed under the Rural Development Project enable us to formulate the following conclusions concerning the basic determinants of Project implementation: 22 • A vast majority of Project tasks are of a key significance for gminas’ functioning. • It should be stressed that gminas’ demand for support in financing investments is authentic – most of these would not be realised without any external support. • Investment activities of the Project are complementary to other programs initiated to increase the standards of living and running business activities in rural areas. These activities complement one another within the frames of the primary objective: development of rural areas. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report 1.9 Analysis of the social and economic effects The scale of State Budget’s benefits resulting from creation of 10265 job placements as a result of RDP, calculated with the use of the methodology updated at the end of 2003 and macroeconomic data, ranged from PLN 47.03 million to PLN 121.66 million, depending on the salary received by the unemployed person taking up a job. The benefit referred to above is only one of many profits. The comprehensive nature of RDP involves a whole spectrum of economic effects. Not all relations can be grasped now, when only few months have passed since Project completion. Direct research enables one to state that economic effects of micro-credits, grants and some services dedicated to the labour market appeared relatively soon. One will have to wait a little longer for complete effects of infrastructural investments and for effects related to selfgovernments’ institutional capacity growth. The greatest time span between involvement of funds and sustainable effects is needed in case of activities dedicated to education. There is no doubt that RDP set up the foundations for a more rapid economic growth of rural areas through: • preparing areas for investments, • improving the efficiency of administration • activating the unemployed, • developing a network of institutions supporting business. Most probably, the growth will be strengthened by assistance funds and these have become much more available to gminas participating in RDP. Even now, the living standards of rural population from the areas covered by RDP have improved noticeably and the first symptoms of increased investment attractiveness translation into concrete activities of business entities can be observed. In effect, many unemployed have been roused from apathy and marasm caused by the long years of unemployment. Even such emergency activities as temporary employment supported many family budgets invaluably. It is worth noting that the costs incurred by the beneficiaries and the Project gminas as their own contribution were not high. As a rule, they were limited to administration costs, cost of travelling and free time. One should make a reservation however that the conclusions presented above, drawn up on the basis of direct research, are of a qualitative nature and do not allow for evaluation of these effects’ scale. When compared with other programs (with respect to particular Components, as no other program corresponding with RDP has ever existed), RDP shows rather favourably, in terms of both cost-efficiency and material productivity as well. What is more, comparison with indicators achieved in open public bidding (based on the Bulletin of Public Procurement), i.e. close to market indicators, has not shown any significant difference either. Thus, one may draw a cautious conclusion (bearing in mind considerable differences between programs being compared) that mechanisms of spending public money were efficient in the Project and that no prodigality took place in terms of statistics (which does not mean, that no isolated cases could be revealed). An analysis of correlation between indicators of Project implementation intensity (the outputs) achieved within the frames of Components A, B1 and C on the territory of particular gminas and powiats and the most important outcomes of the Project, i.e. increase of employment, income and wealth, gives results close to zero in most cases. This leads one to a conclusion that the effect of RDP on the economy of territorial selfConsortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 23 Final Evaluation Study of the Rural Development Project Final Report government units was hardly noticeable until the year 2003. This is a statistical conclusion, certainly, and it is limited to short-term effects. The scale of RDP is one of the main reasons of such a weak translation of the Project into any noticeable macroeconomic effects. Concentration of funds in a single gmina or powiat was too low to change their economic standing radically. But it is not out of the question that similar analyses will show more appreciable results in time, when a multiplication of Project effects occurs. 1.10 Evaluation of Institutional, Organisational and Management Quality Related Solutions of the Project A. Institutions implementing the Project rose to the primary challenge in the form of effective implementation of activities envisaged for the Project. The procedures, especially those related to financial flows – following significant difficulties and delays in the initial period – were modified gradually through cooperation between institutions. The evaluators assess that in the second part of the Project the procedures functioned in an effective and timely manner. The Project was successful in showing representatives of various level of authorities evidently good organisational practices, such as: • comprehensive and continuous monitoring of the Project, • subjecting Project decision-making institutions to control and verification both at the stage of formation and on a current basis in the whole period of activity as well, • setting up mechanisms and incentives to make organisational adjustments (periodical meetings, providing the support of external consultants). Another success was attained in implementation of a project evaluation system based on clear, pre-defined criteria. Many of the respondents regarded it as an anti-corruptive solution, much better than application evaluation systems existing in EU programs now. The long starting period of the Project described in the report of the Superior Chamber of Inspection – should be considered a failure. The delays and negligence observed then can be explained easily – although not justified – with the institutions’ need to implement the new methods of work and to develop mutual compatibility. In consequence, the whole Project stretched on considerably and the schedule of implementing particular Components was not subordinated to Project objectives, but reflected RDP’s ability to overcome organisational problems. B. The World Bank proposed that adjusted procedures of the Flood Emergency Project should be used in RDP. The Bank eliminated the significant weaknesses of the Project, but an important defect which had occurred there remained. The World Bank made a number of steps to eliminate weaknesses of the Flood Emergency Project from RDP. These steps showed their effectiveness in supervision of the RDP funds expenditure and in the course of co-financed undertakings. At the same time, the scheme of implementation was again based on a silent assumption that the conglomerate of institutions appointed to co-operate from the moment when the Project was launched and to implement procedures that were new for them, would be 24 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report able to commence efficient activity immediately. This assumption turned out excessively optimistic twice and as a result, the activities were much behind the original schedules in both cases. C. Co-ordination of activities of various Components was not effective enough. This was caused by a gap in the designed distribution of competence and responsibilities among institutions implementing the Project. As it has been mentioned above, it was Project’s organisational achievement that the institutions involved in it achieved the level of co-operation ensuring completion of their tasks at the basic level, i.e. achievement of quantitative outputs assumed for particular Components. Thus, co-ordination was effected in terms of Project institutions’ ability to collaborate in achievement of the basic objectives. In its wider meaning, co-ordination of activities should be manifested through the ability to perform tasks in a manner enabling mutual strengthening of activities within the frames of one Component and of those performed under various Components. While the capacity to achieve the strengthening effect through the consonance of various activities within the frames of a Component could be seen at the stage of drawing up plans and schedules of implementation, the course of Components showed that the activities were often performed in an accidental sequence and the chance of strengthening the activities by the means of interrelating one with another was missed. In case of activities performed under various Components, it is even more difficult to strengthen them through interaction. To be able to effect this, one needs an entity having sufficient resources at its disposal, a wide and immediate access to information and – first of all – adequate decision-making competence. The scope of this entity’s interest, its mission, should cover co-ordination of Components according to the conception referred to here. The evaluators have not noticed any institutional efforts aimed at co-ordination in this meaning of the word. A question emerges, whether any such entity existed in RDP. This was not the loan provider, although its status in Project supervision was of a key significance. The Project Co-ordination Unit did not play this part – it performed important administrative functions, but despite its central position in the organisational structure, these functions were subsidiary to other subjects. The National Steering Committee should have played the co-ordinating role discussed her. But NSC gathered too seldom and for a too short time to make strategic decisions. The Committee did not interfere much in activities of Components A and B1, while effective co-ordination of activities should involve a continuous critical analysis of the entire Project – to be able to anticipate chances for activities’ synergy and to eliminate any organisational barriers to this synergy. To sum it up, there was no institution in the Project that would have relevant resources, knowledge and authority required to effect co-ordination the scale of entire undertaking. It is worth adding that in the evaluators’ opinion it could have been entirely possible to make the National Steering Committee or even the Project Co-ordination unit perform a task like this effectively. Most probably – on the condition that the Project management system was re-defined and that the selected institution was assigned additional resources (human, first of all) and in case of PCU – the decision-making authority as well. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 25 Final Evaluation Study of the Rural Development Project Final Report D. It is doubtful whether the formula of RDP will ever be used again, but the good practice used in the Project is worth disseminating. The Project organisation had been designed to address: • the decision to structure the Project on a multi-Component basis, • the assumptions concerning participation of regional authorities and local selfgovernments in the Project, • the loan provider’s supervisory function and requirements in the field of monitoring and reporting. This makes RDP a constellation of objectives and institutions appointed to implement these objectives which can hardly be reproduced. At the same time, the Project had two basic weaknesses: • the implementation had commenced before the formation process of institutions intended to implement it was completed; in consequence, the newly established organisations and teams assigned the Project tasks faced the need to develop their own methods of work and to learn the Project procedures, therefore the fact that the timing provided for in the schedule could not be adhered to can not be surprising, as the schedule had not allowed for these needs, • the Project as an entirety had no institutional owner who would take care of more than completion of tasks of one Component. This is why any simple duplication of the entirety of solutions accepted for the Project seems unjustifiable and useful lessons can rather be learned from analysis of mistakes. Nevertheless, it is worth noting the good practices introduced and implanted in participating institutions within the frames of RDP management. The evaluators have identified the following here: 26 • subjecting the institutions – the newly-established in particular – to formal restrictions, such as statute, rules and regulations, specific operating procedures – these were conducive to stabilisation of activities, competence building and corruption counteracting, • the continuous practice of consultations between representatives of various institutions – in the form of working, personal contacts and regular meetings organised by the Project Co-ordination Unit – and also their readiness to disseminate the solutions that have been worked out and to use them in practice as soon as possible. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report 2 SWOT Analysis Table 1. SWOT analysis of the evaluation model selected for the present study Strengths • • • Combination of two approaches: top-down • and bottom-up, enabling one to confront objective data with subjective opinions in order to obtain a more complete picture of • evaluation Precise evaluation criteria • Going beyond the four standard evaluation criteria (relevance, efficiency, effectiveness • and sustainability) and including such aspects as evaluation of attendance, awareness, targeting and sufficiency • Detailed evaluation at each level of the Project: the entire Project, Components, Sub-Components and services • Evaluating not only outcomes and outputs, but the process of implementation and the institutional aspects as well • Evaluation of all significant activities and the whole period of Project implementation • Precise distribution of tasks and issues within the group of experts • Using wide statistical analyses as a basis • Quantitative evaluation supplemented with a comprehensive qualitative assessment (triangulation of methods) • Showing RDP on the background of other similar activities and programs • Addressing the report of the Superior Chamber of Inspection Opportunities • PCU’s co-operation and consultations at • each stage of evaluation • Easy access to documentation facilitated by the Consultant’s internet base • • • Evaluation provides recommendations, shows good and bad experiences that may be used in other development projects in • the future (e.g. NPR, various SOPs, etc.) or other similar activities (e.g. loan funds) Weaknesses It was impossible to cover the long-term impact of RDP Problems with obtaining data from RDP institutions (e.g. concerning the survey made by PBS) Problems with contacting the respondents Problems with obtaining comparable, good quality data about other activities and programs similar to RDP Threats Difficulties with precise and objective identification of net outputs and external outcomes of the Project (respondents’ opinions being the main basis here) Omission of RDP long-term impact may affect the quality of conclusions Evaluation outcomes may determine future decisions to launch similar programs Evaluation outcomes may determine future decisions to launch similar programs Source: Consultant’s own analysis Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 27 Final Evaluation Study of the Rural Development Project Final Report II Introduction Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 29 Final Evaluation Study of the Rural Development Project Final Report The present report has been prepared by the consortium consisting of: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. on the basis of a contract with the Project Co-ordination Unit of the Rural Development Project dated 3 December 2004. 1 Project Description The Rural Development Project was designed at the end of the 90-ies with the idea to support the development of rural areas in Poland and it was expected to result in: • increased off-farm employment, • support for the process of strengthening self-governments of the local and regional level and support for the regional development, • support for the process of building institutional capacity necessary to obtain cofinancing from EU pre-accession and structural funds. With regard to the complex nature of problems experienced by rural areas, multisectoral approach was adopted in RDP. The Project consisted of three Components: A – Microcredit, B – Human Capital Development, C – Rural Infrastructure. Additionally, three SubComponents were distinguished within the frames of Component B: B1 – Labour Redeployment, B2 – Education, B3 – Institution Building in Local and Regional Administration. The multisectoral nature of the Project was based on the assumption that the problem of labour force surplus in rural areas can not be solved within the frames of the sector of agriculture only. This is why the scope of RDP was extended over support for rural entrepreneurship, transformations in the education system and in the labour force structure in rural areas. The area of RDP implementation covered the whole country, but some Components were implemented in selected regions, e.g. Component A – in five voivodships only. The geographical differentiation of RDP particular Components implementation related to selected socio-economic indicators characteristic for particular voivodships. Project implementation was possible due to a loan amounting to EUR 118.8 million provided by the World Bank to the Polish Government. 2 Context of Project Implementation The Rural Development Project was implemented in the years 2000-2005. Its beginning coincided with the worsening of the economic situation in Poland and this was particularly perceptible in rural areas. As a result, the unemployment was growing rapidly there. It is worth stressing that further increase of unemployment was forecasted in that period, as a consequence of reforms expected in the national agriculture. Shortly before RDP was launched, a comprehensive reform of public administration structures had been implemented since January 1999. The changes involved decentralisation of the process of public tasks performance and financing to two new levels of democratically elective self-government authorities: powiats and voivodships. The decentralisation of public tasks and financing to local self-governments made heavy 30 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report demands on the public administration representatives, especially in rural areas, while councillors and board members often had no required organisational and managerial skills. One should not forget that the period when RDP took place was a time of Poland’s preparation for accession to the European Union. Activities were initiated to prepare mechanisms intended to provide effective absorption of pre-accession funds and following integration – structural funds. One of RDO objectives related to building the capacity to absorb the funds. RDP was an element of wider activities aimed at rural areas activation and addressing the Cohesive Structural Policy for Rural Areas and Agriculture Development (1999). The Project continued activities performed under other assistance programs, e.g. the Agricultural Sector Adjustment Loan (ASAL). On the other hand, RDP constituted a source of reach and valuable experience that can be used when planning and implementing further programs, such as SPARD for example. 3 Objective of the Final Evaluation Study of the Project The objective of the present final evaluation of RDP is to: • provide the Government of the Polish Republic with a professional and objective evaluation of the Rural Development Project in terms of effectiveness and efficiency of its objectives achievement; • use the experience and the achievements of RDP in the process of planning and implementing other programs in favour of rural areas, including those co-financed by international financial institutions and/or by EU within the frames of Structural Funds and the Cohesion Fund and • fulfil the provisions of the loan contract specified in the Operational Manual and the Project Appraisal Document. The final evaluation scope covers: • evaluation of RDP objectives achievement based on assumed outcomes and outputs (performance indicators); • economic and financial analysis of RDP; • analysis of external financing effect on investment realisation; • verification of RDP rationale and evaluation of Project outcomes and construction; • analysis of institutional and implementation related assumptions; • Project sustainability and risk aspects; • evaluation of economic efficiency and effect of scale. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 31 Final Evaluation Study of the Rural Development Project Final Report III Methodology Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 33 Final Evaluation Study of the Rural Development Project Final Report 1 Research Model To develop the research model, the Consultant accepted the concept of evaluation defined as: “Independent, objective examination of the background, objectives, results and activities aimed at drawing conclusions that could be used within a wider scope”2. The methodology of the Rural Development Project (RDP) was based on the methodology developed by OECD/DAC and recommended by the European Commission3. According to this methodology, evaluation includes analysis of the so-called Project intervention logic. The intervention logic involves formulation of a sequence of interrelated goals of the project at all levels, i.e.: • operational goals – manifested by outputs, i.e. goods and services produced directly during Project implementation, • specific goals – manifested by outcomes4, i.e. effects achieved immediately after Project completion, • overall goals – manifested by impact, i.e. long-term effects generated by the Project5. Figure 1. Project intervention logic scheme Impact Overall objectives (long-term) Effect (social effects) Outcomes Specific goals (initial effect) Outputs (goods and services produced) Input data (human and financial resources) Project objectives Operational goals Project activities Sources: Consultant’s own analyses 2 MEANS Collection. Evaluating socio-economic Programme, Vol. 6. Glossary, Luxembourg 1999. Evaluation in the European Commision. A Guide to the Evaluation Procedures and Structures currently operational in the Commission’s External Co-operation Programmes, Brussels, March 2001. 4 According to the terminology defined in Evaluating EU Expenditure Programs: A Guide. Ex post and Intermediate Evaluation, Brussels, January 1997, specific goals are called “results”, but we shall keep to the terminology defined in Terms of Reference and, for the sake of reasoning clarity, we shall use the term “outcomes”. 5 Evaluating EU Expenditure Programmes: A Guide. Ex post and Intermediate Evaluation, Brussels, January 1997. 3 34 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report RDP has been evaluated on the basis of five criteria: Relevance – used to evaluate, to what degree Project objectives identify real problems and needs of the target group and respond to them. Efficiency – compares resources (financial, administration, human, etc.) involved in Project implementation with the actual achievements of the Project at the level of output, outcome or impact. Effectiveness – this criterion enables one to assess, whether the objectives defined at the stage of Project designing have been achieved. Comparing the real effects of the Project achieved at the level of output, outcome and impact with the assumptions specified in Project documents, one is able to identify actual achievements of the Project. Utility – confronts the given sector’s / region’s needs and problems with the actual achievements of the Project after its completion. Sustainability – this criterion identifies, whether effects of the Program will last on a midterm and long-term basis, including the question, whether durable positive effects of the Project will last after co-financing ends. In particular, the purpose of evaluation was to give answers to a number of evaluation questions. One of the primary tasks was to answer the question, whether the specific goals and the overall objectives (defining the outcomes and the impact) accepted at the stage of designing the RDP had been achieved as a result of Project implementation. Moreover, it was important to check, how reliably were projections of economic efficiency prepared at the stage of RDP design and to what degree they proved true, particularly in terms of: • accepted assumptions (what they were based on, whether they proved correct as the time proceeded); • methodology (whether computations were carried out according to the state of the art); • expected results (indicators). • efficiency of money engaged in RDP as compared with alternative utilisation by other institutions of programs. It was assumed at the stage of evaluation planning that costs of the Project would be summed up in a breakdown into Sub-Components and the common part (Component D – Project management) allocated to particular Components and Sub-Components. The correctness of cost computation and allocation is a key determinant of the correctness of particular indicators computation and these were supposed to be measured as the unit cost of effect in most cases. Moreover, quantitative effects were to be collected. On this basis, independent computation of Project economic efficiency indicators was supposed to be carried out. Besides, relevant indicators of other, similar, Polish and foreign assistance programs were to be identified – the so-called benchmarks, the Project indicators could be compared with. Programs to be used as a comparative basis were to be identified in consultation with the Client and listed in the Inception Report. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 35 Final Evaluation Study of the Rural Development Project Final Report Where possible, material outputs of particular projects implemented under RDP were to be converted into money, to compute the economic efficiency of the entire Project. The next challenge the evaluation process faced was represented by the task of evaluating the effect of external financing from RDP on investments made by local selfgovernments. The examination was to focus on the effect of the Project in the form of increased number, scale and improved quality of infrastructural investments. The effects was to be measured by the means of comparing efficiency indicators of projects cofinanced under RDP with those financed from other sources. In particular, the development of projects refused co-financing from RDP was to be analysed: • whether they were implemented, • whether they were provided financing from other sources, • what material outputs they produced. Efficiency benchmarks were to be defined for these projects and also for projects that had not apply for RDP financing. The benchmarks would enable one to compare the effects and the influence of financing in particular. Moreover, the effect of delays in disbursement of Project funds on project implementation was to be assessed, especially the related additional costs incurred by gminas. Besides, the evaluation was supposed to cover the efficiency of Project assumptions concerning multisectoral activities, geographical diversification of activities, the “added value” and public participation in the Project. This task was of a horizontal nature and it included evaluation of Components A, B1, B2,B3, C in terms of: • multisectoral aspect – in the meaning of combination of the sector of education, entrepreneurship in rural areas, employment of the unemployed, education, public administration, • public participation, • geographic diversification of activities, • World Bank’s added value. In particular, it was planned to check, to what degree the above assumptions had been addressed at the stage of particular Components implementation and what results this approach produced. The next evaluation task was to assess institutional, organisational and management quality related solutions of the Project. In particular, the effectiveness of institutions participating in Project implementation was to be analysed, i.e.: • 36 whether the institutions concerned implemented and used correct procedures consistently, fully complying with the guidelines; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report • the procedures in force were to be verified in terms of their completeness and internal consistency; moreover, distribution of authority and responsibilities among persons participating in the Project were to be checked and the degree to which the procedures had provided for various variants of actions; whether there were any situations that had not been envisaged in original procedures and whether the procedures were updated when needed; • qualifications, quantity and availability of personnel participating in Project tasks implementation were to be checked; • it was planned to check, whether material resources intended for performance of tasks had been selected in an optimal manner – sufficient and not in excess; • it was to be checked, whether the procedures had been used commonly and equally in all institutions involved in a given task; should any differences be revealed, their background was to be checked; • various institutions’ ability to co-operate was to be checked. It was planned to look for answers to the following questions: had these institutions complementary resources at their disposal, for example customised IT solutions; had there been any delays in Project tasks that might have resulted from the lack of synchronisation of cooperating institutions’ activities; had the co-operating institutions co-ordinated the process of introducing changes to procedures; • it was supposed to be checked, how institutions involved in the Project had responded to problems and needs reported to them by other stakeholders, including the beneficiaries – had they been analysed and what their effect had been; • it was planned to check the institutions’ ability to learn, i.e. to record and analyse problems and to design and implement corrective actions, to assess the effectiveness of corrective actions, to develop models of actions considered to be the best practice and to disseminate them among institutions performing the same tasks. The evaluation was planned to include the assessment of utility of procedures, criteria, institutional solutions, etc. accepted in RDP for future programs. It was assumed that apart from direct outcomes, RDP would leave a certain “heritage” in the form of institutions, procedures, trained personnel and habits that would become useful when implementing other programs, especially programs and projects financed from structural funds. Thus, it had to be examined and assess, to what degree RDP had become a foundation for structural funds, particularly in terms of: goals, procedures, beneficiaries selection criteria, formation of institutions, staff education. Moreover, it was proposed to include risk management assessment in Project evaluation. PAD had identified and assessed critical aspects of the Project risks. Methods of minimising those had also been provided. It was the evaluators’ task to assess whether the risks had been assessed correctly and whether they had been prevented effectively. Another purpose of evaluation was to assess the use of economic efficiency criterion and of the effect of scale criterion by Project beneficiaries. These effects could be achieved first of all through establishing unions of gminas applying jointly for financial support from the Project funds and implementing joint ventures. Thus, it was supposed to check, whether such unions had been created and if not - why. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 37 Final Evaluation Study of the Rural Development Project Final Report The evaluation was supposed to include assessment of public procurement procedures used in the Project. Research work and analyses planned for this section were expected to cover: 2 • the influence of standard bidding and contract procedures for consultants on institutional adaptation to absorption of pre-accession funds (compliance with EU requirements concerning bidding procedures); • potential problems that might have emerged as a result of differences between World Bank Requirements, EU requirements and Polish requirements; • elements of the Polish law related to public procurements that might have been used in the Project directly. Research Work and Data Collecting To be able to carry out a complete assessment of Project intentions and achievements, the evaluators planned to use relevant secondary and primary sources. 2.1 Secondary Sources Indicators relevant to evaluation were analysed on the basis of information collected in the Consultant’s database, originating from such sources as inter alia: RDP implementation reports, MIS, Main Statistical Office, regional institutions (voivodship Administration Offices, Marszlek Offices, Boards of Education, Road Administration Boards, analyses of local documents characteristic for particular areas). A comparative analysis of performance against the planned values was carried out and – where justifiable and possible – selected voivodships, powiats and gminas where particular elements of the Project had not been implemented were used for comparison. As a result of comparisons and analyses, the degree of planned performance indicators achievement by particular Components of the Project was assessed. When analysing the effects of the Project, the evaluators analysed – inter alia – the following secondary sources: • materials, documents and sets of data (e.g. MIS, MIS for Sub-Component B1) produced during RDP implementation, • statistical database POLSKA 2004, • data available from the Main Statistical Office (Regional Data Bank, statistical yearbooks, information from the National Census and the Agricultural Census), • data available from Voivodship Labour offices and Powiat Labour Offices. 2.2 Primary Sources Primary sources were analysed for exploration and illustration purposes. These analyses covered specific issues related to particular Components. 38 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Interviews were made in order to obtain two types of information: • information enabling one to follow the effects at the most basic level (from beneficiaries’ point of view), • information supplementing analyses of secondary sources data with more detailed data, observable at the level of beneficiaries’ assessment and opinions only and also at the level of institutions co-operating in Project implementation. 2.2.1 Research Tools and Techniques The research work based on collecting information from primary sources was carried out mainly by the means of face-to-face interviews. This method of collecting data enabled the evaluators to meet the expectation included in clause 5.1 of Terms of Reference best – i.e. to approach all beneficiaries of RDP. On the other hand, it provided the required quality and reliability of data. Depending on the nature and the expected structure of information to be obtained, the research work had the form of: • in-depth interviews or expert interviews, • questionnaire surveys (in special cases – questionnaires filled by respondents on their own). Both qualitative and quantitative surveys were carried out with the use of research tools developed to address – on the one hand – Project-specific needs and on the other hand – rules and conditions imposed by the good practice, controlling the way how questions should be asked, enabling one to obtain sincere and concrete answers and to avoid mistakes and traps resulting from the construction of research tools. The research tools prepared for the Project were presented to PCU before the survey commenced, to provide insight into their contents and form and the possibility to make any comments or to clarify any concerns. In-depth or expert interviews In-depth interviews involved a conversation with a respondent. The conversation concentrated on selected topics and was conducted according to a pre-designed interview scenario, but it enabled the respondent to make relatively loose and unstructured statements. The purpose was to collect information that would be hard to obtain by the means of classical questionnaire-based interviews – when both the respondent and the interviewer needed more freedom. Answers were often received to questions that could never be asked in a standard (questionnaire) mode. These applied to more complex issues – such as implementation of procedures, identification of weaknesses and strengths of what had been offered under particular Components, etc. Individual in-depth interviews were conducted by specially prepared sociologists and psychologists experienced in this type of research work. Some of the individual in-depth interviews had a form of expert interviews: they were conducted by experts co-operating on evaluation and were supplemented with expert consultations (the expert developed his/her opinion on specific issues immediately during the interview) and – where justifiable – with analysis of documents. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 39 Final Evaluation Study of the Rural Development Project Final Report Questionnaire-based interviews (in special cases – questionnaires to be filled by respondents on their own) Questionnaire-based interviews were used for collecting data that were thereafter subjected to quantitative analyses. They were conducted on the basis of questionnaires compiled specially for the purpose of the present project. They contained both closed and open-ended questions. The questions were about facts that had taken place during particular tasks and about respondents’ opinions about particular tasks of the entire Project. Besides contentsrelated questions, the questionnaires contained also a specification: in case of individuals – a set of socio-demographic information and in case of companies and institutions – the most important information concerning location, size, employment, etc. Quantitative surveys were effected by the means of face-to-face interviews in their majority. Some of the tools (in case of Component C) were prepared in a manner that made it possible to use them as questionnaires to be filled by respondents on their own. An assumption was made that in case of data that did not involve opinions, judgements or experience, but rather purely reporting indicators or information, the interviewer was allowed – if this was respondent’s wish – to leave the questionnaire and to come back for it at the time agreed. Checklists for in-depth interviews Additional auditing was carried out during meetings with respondents, to supplement indepth interviews conducted by institutional auditors. The list of audit steps was presented in appendix 2 to the Inception Report. It had a form of a checklist which was given to each auditor and the purpose of which was to make it easier for the auditor to control the audit meeting. Certainly, each audit meeting was dedicated first of all to the in-depth interview. During the interview or after it, the auditor was expected to check the items from the checklist. These related mainly to availability of resources necessary to perform the Project tasks. Thereafter, the auditor updated relevant data. 2.2.2 Selection of Samples At the stage of contract execution, the Consortium supplemented the methodology with the rules of selecting samples for the questionnaire survey. Selection of respondents to in-depth interviews In-depth interview covered the total of 295 respondents who represented central, voivodship, powiat and gmina institutions and subjects supporting Project implementation at the central and voivodship level. Some of the interviews were made with Project beneficiaries. In case of institutions, the sample was selected in co-operation with the institutions and the companies concerned. Lists of persons were agreed, with the following characteristics taken into consideration: (1) the longest time of work for RDP, (2) the position in the hierarchy of respondent’s institution enabling him/her to provide information and formulate opinions resulting from the in-depth interview scenario, (3) specialisation and qualifications adequate to interview scenario requirements (e.g. providing professional services to Project beneficiaries, operating databases set up by the institutions concerned, supervision of the Project finance. 40 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Selection of respondents to questionnaire-based interviews At the stage of proposal writing it had been assumed that two methods of sample selection would have been used for the purpose of the present study. Although this assumption did not change, it is important that as a result of absence of an integrated system of information about beneficiaries containing address data, the sample selection procedure was much complicated and much more time was needed to prepare the survey and the very process of survey was more complicated as well. Assumptions for selection of respondents at the stage of proposal writing According to clause 5.2 of Terms of Reference, the survey samples were selected to cover all groups of beneficiaries, but also – additionally – persons involved in Project implementation and being a valuable source of information. The samples covered all voivodships where particular Components of the Project had been implemented (except for the samples for Sub-Component B1, where the survey had been made in two voivodships selected on purpose). According to the assumptions presented in the proposal, the selection was carried out as follows: • Beneficiaries and persons co-operating in Project implementation: stratified random samples, based on databases containing phone and address data of RDP beneficiaries or persons co-operating in RDP implementation. Additionally, in case of final beneficiaries whose phone address data were not in Client’s possession, the sample drawing was effected at the level of powiats or gminas whose help was thereafter necessary to find out, how potential beneficiaries could be contacted (see: beneficiaries of Sub-Component B2). • Comparative samples – these were selected in a manner tailored for each particular Component and each part of the survey, to be able to obtain information that can be now compared with the information collected from Project beneficiaries and from institutions co-operating in implementation. Operational database construction When PBS commenced work on preparing the survey operational database, it turned out that no integrated system of information about beneficiaries existed at the level of the entire Project or at the level of particular Components. Unfortunately, such information was a necessary condition of efficient drawing of gminas and respondents to be surveyed. This situation did not result from any mistakes made at the stage of Project implementation, but only from the fact that the issue had not been thought over or planned at the stage of Project planning. Therefore, it is worth assuming in any future programs that an integrated system of information about final beneficiaries should be built with phone and address data and also – resources permitting – with any other interesting indicators that may enable one to perform some part of evaluation without the need of conducting face-to-face interviews with final beneficiaries. When designing the selection of samples, apart from the duties accepted under the contract, the Evaluator had to create the information system referred to. This fact caused a considerable delay in commencement of the main survey, this situation representing a significant problem in the context of delay in the whole evaluation commencement. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 41 Final Evaluation Study of the Rural Development Project Final Report To sum up the stage of IT system construction for the purpose of the present evaluation, one should pay attention to the following facts: 1) At the stage of RDP planning assumptions were made concerning indicators that should be analysed during evaluation, but no measures were taken to organise a system of collecting data about the Project outcomes and beneficiaries to enable the evaluators to draw samples from a few basic electronic databases without problems – in a minimum version – or, to monitor changes at the level of key statistical indicators – in the maximum version – at the stage of evaluation. 2) The survey was easiest in case of Component A, where the implementing institutions (Rural Development Foundation and Foundation for the Development of Polish Agriculture) provided phone and address databases of beneficiaries. In practice, no problems emerged there, apart from typical information that some potential beneficiaries had changed the address or left home for a longer time. SubComponent B3 and Component C did not require many additional activities of Evaluator to construct the sample (the samples were created on the basis of documentation provided by PCU and ARMA). 3) In case of surveying beneficiaries of Sub-Component B1, it was necessary to obtain information from each voivodship covered by this Sub-Component separately. Another important problem arose when acceptance of personal data processing had to be obtained in contracts the beneficiaries had been obliged to sign. In most cases, Voivodship Labour Offices had no knowledge of the matter or were forced to contact the Contractors and to obtain confirmation from them directly. When this issue was being checked, it turned out that no uniform contract for beneficiaries with a clause like this had been used and that the contents of contracts had depended on the Contractor. The absence of a written acceptance of personal data processing constituted a basis of refusal to provide beneficiaries’ phone and address data to the Evaluator and as a result it was impossible to effect the planned survey. Finally, the problem was solved through a reference to the provisions of the Act of Personal Data Protection (Journal of Laws 97.133.883, amendments: Journal of Laws 00.12.136, Journal of Laws 00.50.580) – art. 23 para. 4 and 5 – relation between the Evaluation and the base Administrator’s discharge of tasks performed for the public benefit, where no rights and freedom of persons the data apply to are violated; art. 26 par. 2, which provides that processing data for a purpose which differs from the purpose for which data was collected is admissible, unless it violates the rights and the freedom of the person the data apply to and is performed for scientific, didactic, historical or statistical purpose, on the conditions specified in art. 29 para 1 – 4 of the same Act. 4) Similarly, the process of constructing operational database for drawing a relevant sample for the purpose of Sub-Component B2 evaluation encountered significant problems. The Client provided a list of IT training beneficiaries and a list of refurbished schools to the Evaluator, but specification of information about the remaining activities (pre-school classrooms equipment, school network consolidation, didactic aids purchasing, didactic training for teachers and school directors) had a form of a list of gminas that had applied for co-financing, instead of a list of schools concerned. As a result, additional resources had to be used, although this had not been envisaged on the basis of information provided by PCU at the stage of Evaluation planning. 42 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Table 2. Tool number Specification of research tools used for evaluation – questionnaire-based interviews and in-depth interviews; general information about selection of respondents Description of respondents Number of interviews Method of interviewing assumed conducted Other comments General assumptions and indicators for the entire Project (ToR, 4.3) 1 Representatives of RDP gminas Questionnaire-based interviews 100 100 2 Representatives of non-Project gminas Questionnaire-based interviews 100 100 Specific assumptions and indicators for particular Components (ToR, 4.4) 3 Component A: loan advisers In-depth interviews (IDI) 5 5 4 Component A: beneficiaries In-depth interviews (IDI) 5 5 5 Component A: beneficiaries Questionnaire-based interviews 300 300 6 Component A: potential beneficiaries not Questionnaire-based covered by the Project (similar to beneficiaries) interviews 300 301 7 Sub-Component B1: consultants well-versed in In-depth interviews (IDI) labour market analyses 7 6 8 Sub-Component B1: persons involved in In-depth interviews (IDI) Component implementation at the VLO level 7 7 9 Sub-Component B1: persons from Powiat Questionnaire-based Labour offices involved in Sub-Component B1 interviews implementation 100 103 10 Sub-Component individuals) 700 706 11 Sub-Component B1: Sub-Component project Expert interviews manager at the level of Ministry of Economy and Labour Project Team 1 1 12 Sub-Component B1: persons from non-Project Questionnaire-based gminas, similar to beneficiaries interviews 400 423 B1: beneficiaries (private Questionnaire-based interviews Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 43 Included in tool no 37 Final Evaluation Study of the Rural Development Project Final Report Tool number Number of interviews Method of interviewing assumed conducted 13 Sub-Component B2: VPIU representatives – In-depth interviews (IDI) persons involved in education Component implementation 7 7 14 Sub-Component B2: members of gmina In-depth interviews (IDI) authorities competent in education related matters 24 24 15 Sub-Component B2: trained teachers Questionnaire-based interviews 600 602 16 Sub-Component B2: trained school directors Questionnaire-based interviews 300 298 17 Sub-Component B2: directors of schools where Questionnaire-based software was provided interviews 150 151 18 Sub-Component B2: directors trained in IT 300 303 19 Sub-Component B2: directors of refurbished Questionnaire-based schools interviews 40 42 20, 21 Sub-Component B2: school directors – non- Questionnaire-based Project schools interviews 40 40 Sub-Component B3: members of authorities Expert interviews from gminas that applied to IDP 40 40 22 44 Description of respondents teachers and school Questionnaire-based interviews Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Other comments Interviews divided into two groups (22a and 22b): IDP participants (20 persons) and participants of IDP Dissemination Program (20 persons) from Lubelskie and Kujawsko-Pomorskie voivodships who participated in the dissemination Program – regional conferences Final Evaluation Study of the Rural Development Project Final Report Tool number Description of respondents Number of interviews Method of interviewing assumed conducted 23 Sub-Component B3: representatives of NGOs Expert interviews situated in Project gminas 20 0 24 Sub-Component B3: representatives of Questionnaire-based territorial self-government units where training interviews was provided 60 60 25 Sub-Component B3: households in Project questionnaire-based gminas interviews 300 300 26 Sub-Component B3: micro-enterprises in IDP questionnaire-based gminas interviews 30 30 27 Sub-Component B3: households in non-Project questionnaire-based gminas interviews 300 301 28 Sub-Component B3: micro-enterprises in non- questionnaire-based Project gminas interviews 30 30 29 Component C: members of authorities in In-depth interviews (IDI) gminas where infrastructural investments were effected 40 40 30 Component C: members of authorities in questionnaire-based 100 gminas where infrastructural investments were interviews or a effected questionnaire filled by respondents on their own Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 96 45 Other comments Questions about NGOs were included in in-depth interviews with IDP participants and it was stated on this basis that it was impossible to identify NGOs as a participant of activities performed within the frames of training programs or of IDP. It was a problem to identify the legal status of organisations operating in rural areas – Village Women’s Clubs and Voluntary Fire Brigades. Most often, gminas informed that there were no NGOs on their territory they co-operated with, therefore it was necessary to eliminate this element of survey. Final Evaluation Study of the Rural Development Project Final Report Tool number Description of respondents Number of interviews Method of interviewing assumed Other comments conducted 31 Component C: households in gminas where questionnaire-based infrastructural investments were effected interviews 400 400 32 Component C: representatives of enterprises in questionnaire-based Project gminas interviews 100 99 33 Component C: members of authorities in In-depth interviews (IDI) gminas not covered by Component C 40 40 Sample divided into 2 groups (33a and 33b): persons who implemented a project without RDP funds and persons who did not implement a project without RDP funds Examination of the impact of external financing on investment projects implementation, consequences of providing / not providing RDP financing (ToR, 4.6) 34 Component C: representatives of gmina questionnaire-based 100 authorities responsible for implementation of interviews or a projects supported under Component C questionnaire filled by respondents on their own 96 35 Component C: representatives of gmina authorities responsible for implementation of projects supported under other assistance programs questionnaire-based 100 interviews or a questionnaire filled by respondents on their own 96 36 Component C: representatives of gmina questionnaire-based 100 authorities from gminas that applied for interviews or a Component C co-financing, but were refused it questionnaire filled by respondents on their own 97 Included in the sample for tool 30 Verification of the grounds for, and the assessment of the results: multi-sectoral nature, geographical differentiation, active participation (bottom-up approach, “added value” that has resulted from World Bank support to the Project (ToR, 4.7a) 37 Persons managing particular Components (A, Expert interviews B1, B2, B3, C), persons from ministries, Agency for Reconstruction and Modernisation of Agriculture and BGK 10 38 Authorities of territorial self-government units 100 In-depth interviews Analysis of institutional and implementing arrangements (ToR, 4.7b) 46 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 10 Carried auditors out by institutional Included in tools no 14, 22a, 22b and 29 Final Evaluation Study of the Rural Development Project Final Report Tool number Description of respondents Number of interviews Method of interviewing assumed conducted 39 Institutions managing the Project or Expert interviews participating in its implementation at the central level (project teams in particular ministries) 10 10 40 NSC Expert interviews 5 5 41 PCU Expert interviews 5 6 42 ARMA research team Expert interviews 2 2 43 Five RSCs – two persons from each Expert interviews 10 10 44 Ten VPIU – one person from each Expert interviews 10 10 45 Five VLO – one person from each Expert interviews 5 10 46 Ten ARMA RD – one person from each Expert interviews 10 10 47 Companies or institutions providing services to Expert interviews Project stakeholders 20 20 Other comments Analysis of Project sustainability and risk aspects (ToR, 4.7c) 48 Representatives of institutions responsible for Expert interviews implementation of structural funds, i.e. first of all: Ministry of Economy and Labour, Ministry of Agriculture and Rural Development, Ministry of Environment and other, to identify the procedures, the people and the institutions that could be “inherited” by them from RDP 10 10 49 Officers from institutions responsible for RDP Expert interviews implementation – to assess their suitability (education, experience) after Project completion 10 10 50 RDP beneficiaries – to what degree the In-depth interviews experience they acquired will facilitate their expert interviews efforts to obtain support from pre-accession and structural funds or 100 100 Economic efficiency and economy of scale (ToR 4.7d) 51 VPIU representatives expert interviews 7 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 7 47 Included in tools no 14, 22a, 22b and 29 Final Evaluation Study of the Rural Development Project Final Report Tool number 52 Description of respondents Number of interviews Method of interviewing Representatives of gminas and powiats questionnaire-based incorporated and not incorporated in gmina interviews unions assumed 100 Source: Consultant’s own analyses 48 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. conducted 96 Other comments Included in tool no 34 Final Evaluation Study of the Rural Development Project Final Report Table 3. Tool number Description of the sample drawing process for the purpose of questionnaire-based interviews. Description of respondents Operational base from which the sample was drawn Sample drawing method The outcome and impact indicators for the entire Project (ToR, 4.3) 1 (W_1) Representatives of gminas Operational base: list of all included in the Project gminas included in RDP, with the number of Components particular gminas participated in taken into account; constructed on the basis of Project documentation A two-degree selection scheme has been used. At the first stage, gminas were drawn in voivodship strata. The process of drawing was independent in each voivodship and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the number of Project gminas represented in the overall population of Project gminas. Simple drawing was effected, with an equal probability of selection. At the second stage, persons from gmina administration offices were selected with a focus of their involvement in the Project and the ability to provide competent information about the situation in gmina. 2 (W_2) Representatives of gminas Operational base: gminas not not included in the Project – included in the Project or comparative sample included to an insignificant degree; constructed on the basis of Project documentation. It was assumed at the stage of survey planning that the comparative sample should consist of gminas that had not been included in the Project. It turned out at the stage of analysing the population of Project and non-Project gminas that there were only 16 gminas all over Poland that had been eligible for the Project but had not been included in any of the Project activities. It was determined on the basis of consultations with the Client that the pool of 84 missing gminas would be provided through including those gminas whose participation in the Project had been “of little intensity”, where the type of activities that had been implemented gave a high certainty that these had no significant impact on gmina’s condition. Thus, gminas meeting the following criteria were selected for this part of the sample: not more than 10 persons had been provided support under Component A, OR not more than 10 persons had participated in IT training, OR co-financing for equipping pre-school classrooms had been provided under Sub-Component B2 only (a single investment), OR co-financing for the maximum of 3 refurbishing or modernisation undertakings in schools / school common rooms had been provided under Sub-Component B2. Drawing was not necessary in this part of the survey project, as it covered the whole population in fact. Interviews were conducted with these persons from gmina offices who were able to provide competent information about the situation in gmina. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 49 Final Evaluation Study of the Rural Development Project Final Report Tool number Description of respondents Operational base from which the sample was drawn Sample drawing method Assumptions and indicators concerning particular Components (ToR, 4.4) 5 (A_1) Component A: beneficiaries Operational base: a list of A multi-degree sample drawing scheme was used. beneficiaries provided by the At the first stage of sample drawing, powiats were drawn. The drawing process was effected in Client. each voivodship independently, with the probability proportional to the number of beneficiaries and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the given stratum constituted in the population. At the second stage, gminas in powiats were drawn. Similarly as at the first stage - with the probability proportional to the number of persons participating in the Project. At the last stage of sample drawing, beneficiaries were drawn in gminas – with an equal probability of selection. 6 (A_2) Component A: comparative sample – persons eligible in terms of social and demographic criteria, who had not benefited from the Project Operational sample: the survey conducted in voivodships included in RDP; address sample; respondents selected from among persons meeting formal criteria of participation in Component A A multi-degree sample drawing scheme was used. At the first stage, gminas in voivodship strata were drawn. The process of drawing was independent in each voivodship and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the voivodship’s participation in the Component. Simple drawing was effected, with the probability of selection proportional to the number of beneficiaries. At the second stage, starting points were selected automatically – district electoral commissions selected with the probability proportional to the number of people with the voting rights. At the next stage, the random route method was used for selecting private addresses, where a respondent meeting the survey eligibility criteria was selected purposefully (a person eligible for a micro-credit, who had not been granted it). 9 (B1_1) Sub-Component B1: persons from Powiat Labour Offices involved in implementation of Sub-Component B1 Operational sample: a list of gminas/powiat where SubComponent B1 had been implemented It was assumed at the stage of survey planning, that interviews of part of will be conducted with representatives of territorial self-government units, where Sub-Component B1 had been implemented. Since it turned out that representatives of TSUs had no information about Component implementation and were not able to answer the questions, the sample was redefined after consultations with PCU and this part of the survey project was transferred to Powiat Labour Offices. The survey covered the whole population of Powiat Labour Offices situated on the territory of Sub-Component B1 implementation (130 PLO). In powiats participating in the Component, persons were selected from Powiat Labour Offices with a focus on their knowledge of powiat’s situation within the scope required for the survey. 50 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Tool number 10 (B1_2) 12 (B1_3) 15 (B2_1) 16 (B2_2) Description of respondents Sub-Component beneficiaries individuals) Operational base from which the sample was drawn B1: Operational base: a list of (private private beneficiaries constructed on the basis of phone and address data obtained from Voivodship Labour Offices Sample drawing method A multi-degree sample drawing scheme was used. At the first stage, a purposeful selection of powiats from Małopolskie and Kujawsko-Pomorskie voivodships took place (those powiats were selected, where most activities of Sub-Component B1 had been implemented). At the second stage, individual beneficiaries from the selected powiats were drawn with the equal probability of selection. Beneficiaries were drawn on the basis of address lists possessed by Voivodship Labour Offices. Operational base: the survey effected in voivodhips included in RDP; random sample; respondents selected from among persons formally eligible for SubComponent B1 A multi-degree sample drawing scheme was used. Sub-Component B2: teachers Operational base: a list of who had participated in gminas where teachers had didactic training participated in didactic training; when beneficiary gminas were drawn – interviewers contacted individual respondents through those gmina administrations that had applied for RDP support. A multi-degree sample drawing scheme was used. Sub-Component B1: comparative sample: – persons eligible in terms of social and demographic criteria, who had not benefited from Sub-Component B1 Sub-Component B2: school Operational base: a list of directors who had participated gminas where school in didactic training directors had participated in didactic training; when beneficiary gminas were drawn – interviewers contacted individual respondents through those gmina administrations that had applied for RDP support. At the first stage, a purposeful selection of powiats from Małopolskie and Kujawsko-Pomorskie voivodships took place. At the second stage, gminas were drawn in the selected powiats, with the probability of selection proportional to the number of inhabitants. At the last stage, a respondent meeting the criteria of participation in Component was selected. At the first stage, gminas were drawn with the probability proportional to the number of beneficiaries. At the next stage, schools were selected (on the basis of data from gmina administration offices) where the beneficiaries worked. At the last stage, employees who had attended training were selected (teachers and school directors). A multi-degree sample drawing scheme was used. At the first stage, gminas were drawn with the probability proportional to the number of beneficiaries. At the next stage, schools were selected (on the basis of data from gmina administration offices) where the beneficiaries worked. At the last stage, employees who had attended training were selected (teachers and school directors). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 51 Final Evaluation Study of the Rural Development Project Final Report Tool number 17 (B2_4) Description of respondents Operational base from which the sample was drawn Sub-Component B2: directors Operational base: a list of of schools that had been gminas where schools had provided software been provided co-financing for software; when beneficiary gminas were drawn, interviewers contacted individual school directors through gmina administration offices that had applied for RDP support Sample drawing method A multi-degree sample drawing scheme was used. At the first stage of sample drawing, powiats were drawn. The drawing process was effected in each voivodship independently, with the equal probability of selection and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the given stratum constituted in the population. At the second stage, gminas in powiats were drawn with the equal probability of gmina selection from the powiats that had been drawn. On the basis of data obtained from gmina administration offices, Project schools were selected at the third stage. The interviews were conducted with school directors. 18 (B2_3) Sub-Component B2: teachers Operational base: a list of A multi-degree sample drawing scheme was used. and school directors who had beneficiaries provided by the The operational base consisted of individuals who had attended IT training. Respondents were been trained in IT Client selected by the means of simple sample drawing in voivodship strata and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the given stratum constituted in the population. 19 (B2_5) Sub-Component B2: directors Operational base: a list of of renovated schools gminas where schools had been provided co-financing for refurbishing; when beneficiary gminas were drawn, interviewers contacted individual school directors through gmina administration offices that had applied for RDP support 20, 21 Sub-Component B2: (B2_6) comparative sample: school directors – non-Project schools A multi-degree sample drawing scheme was used. At the first stage of sample drawing, powiats were drawn. The drawing process was effected in each voivodship independently, with the equal probability of selection and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the given stratum constituted in the population. At the second stage, gminas in powiats were drawn with the equal probability of gmina selection from the powiats that had been drawn. On the basis of data obtained from gmina administration offices, Project schools were selected at the third stage. The interviews were conducted with school directors. Operational base: a list of A multi-degree sample drawing scheme was used. gminas that had not been included in the Project; At the first stage of sample drawing, gminas were drawn. The drawing process was effected in each voivodship independently, with the equal probability of selection and the method assumed random selection that the number of interviews in a voivodship should be proportional to the fraction of schools the given stratum constituted in the population. At the second stage, non-Project schools were selected. The interviews were conducted with school directors. 52 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Tool number 24 (B3_1) Description of respondents Sub-Component B3: representatives of territorial self-government units where training had been provided Operational base from which the sample was drawn Sample drawing method Operational base: a list of A 2-degree sample drawing scheme was applied. gminas where training had been provided; random The operational base consisted of a list of Project gminas. The selection was effected by the means of simple drawing in voivodship strata and the method assumed that the number of selection interviews in voivodships should be proportional to the fraction of beneficiaries (gminas) the given stratum represents in the population At the second stage, a participant of the Project training was selected. 25 (B3_2) Sub-Component B3: Operational base: a list of households in Project gminas gminas where training had been provided; random selection; address starting points were used on a given gmina territory A multi-degree sample drawing scheme was used. At the first stage, gminas were drawn. The process of drawing was independent in each Project voivodship with the probability proportional to the number of inhabitants and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the given stratum represented in the population. At the second stage, starting points were selected – district electoral commissions selected with the probability proportional to the number of people with the voting rights. At the next stage, the random route method was used for selecting private addresses, where a respondent was selected at the last stage. Respondents were selected purposefully from among persons who had at least one contact with the local gmina administration office within the last year (2004). 26 (B3_4) Sub-Component B3: micro- Operational base: a list of A multi-degree sample drawing scheme was used. enterprises in Project gminas gminas where training had been provided; purposeful At the first stage gminas participating in IDP were drawn. The sample was drawn with the probability proportional to the number of registered business entities with the employment not selection larger than 9 people. At the second stage, enterprises were selected from among those who had contacts with the gmina administration office in the year 2004. Respondents were selected from among business owners or managers who had at least one contact with the local gmina administration office within the last year (2004). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 53 Final Evaluation Study of the Rural Development Project Final Report Tool number 27 (B3_3) Description of respondents Sub-Component B3: comparative sample: households in gminas not participating in IDP Operational base from which the sample was drawn Operational base: a list of gminas not participating in IDP; random selection; address starting points were used on a given gmina territory Sample drawing method A multi-degree sample drawing scheme was used. At the first stage of sample drawing, powiats were drawn. The drawing process was effected in each voivodship independently, with the equal probability of selection At the second stage, gminas in powiats were drawn with the probability proportional to the number of inhabitants. At the next stage, starting points were selected – district electoral commissions selected with the probability proportional to the number of people with the voting rights. As the next step, the random route method was used for selecting private addresses, where a respondent was selected at the last stage. Respondents were selected purposefully from among persons who had at least one contact with the local gmina administration office within the last year (2004). 28 (B3_5) 30 (C_1) Sub-Component B3: comparative sample: microenterprises in gminas not included in the SubComponent Operational base: a list of gminas not participating in IDP (similar to gminas participating in IDP); purposeful selection Component C: representatives of gmina administration offices, where infrastructural investments had been effected Operational base: a list of gminas participating in Component C, quota-based selection (with types and number of activities performed in a given gmina taken into account) A multi-degree sample drawing scheme was used. At the first stage of sample drawing, powiats were drawn. The drawing process was effected in each voivodship independently, with the probability of selection proportional to the number of business entities. At the second stage, gminas in powiats were drawn with the probability proportional to the number of inhabitants. At the third stage, starting points were selected – district electoral commissions selected with the probability proportional to the number of people with the voting rights. At the last stage, respondents were selected purposefully from among business owners or managers persons who had at least one contact with the local gmina administration office within the last year (2004). A multi-degree sample drawing scheme was used. At the first stage of sample drawing, gminas were drawn. The drawing process was effected in each voivodship independently and the method assumed that the number of interviews in a voivodship should be proportional to the fraction of beneficiary gminas, the given stratum constituted in the population. Simple sample drawing was effected, with the equal probability of selection. At the second stage, gminas were drawn with the equal probability of selection. Respondents were selected in gmina offices purposefully from among persons who were competent in matters related to Component C implementation. 54 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Tool number 31 (C_4) 32 (C_5) Description of respondents Component C: households in gminas where infrastructural investments had been effected Component C: representatives of enterprises in gminas included in the Program Operational base from which the sample was drawn Sample drawing method Operational base: a list of gminas participating in Component C, quota-based selection (with types and number of activities performed in a given gmina taken into account); random selection of respondents A multi-degree sample drawing scheme was used. Operational base: a list of gminas participating in Component C, gminas selected on a quota basis (with the types and numbers of activities performed in gmina taken into account); purposeful selection of respondents A multi-degree sample drawing scheme was used. At the first stage of sample drawing, gminas were drawn. The drawing process was effected in each powiat covered by Component C independently, with the probability of selection proportional to the number of investments financed under the Project. At the second stage respondents were selected (purposeful selection) – persons in households (second degree drawing unit), competent in matters related to investments effected in the given gmina (is or was a decision-maker in matters relating to the use of infrastructural developments). At the first stage of sample drawing, powiats were drawn. The drawing process was effected in each voivodship independently, with the probability of selection proportional to the number of beneficiary gminas and the method assumed that the number of interviews in a voivodship should be proportional to the fraction the given stratum constituted in the population. At the second stage, gminas in powiats were drawn with the equal probability of selection. At the last stage, representatives were selected in enterprises launched during the Projectfinanced infrastructural investment implementation or after its completion. Examination of the impact of external financing on implementation of investment projects, consequences of being or not being provided with RDP financing (ToR 4.6) 34 Component C: representatives of gmina authorities responsible for implementation of projects cofinanced under Component C Operational base: a list of Included in the sample for tool 30 gminas participating in Component C, gminas selected on a quota basis (with the types and numbers of activities performed in gmina taken into account); 35 (C_2) Component C: comparative sample: representatives of gmina authorities responsible for implementation of projects co-financed under other assistance programs Operational base: a list of A two-degree sample drawing scheme was applied. gminas participating in Component C, purposeful Gminas were drawn with the equal probability of selection in voivodship strata and the method assumed that the number of interviews in a voivodship should be proportional to the fraction of selection. gminas from SAPARD, the given stratum represented in the population (of gminas covered by SAPARD co-financing). Only gminas meeting the size-related criterion of participation in RDP were taken into consideration – rural and mixed rural-urban gminas with the population below 20 thousand. At the second stage, a gmina administration representative was selected (Project co-ordinator). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 55 Final Evaluation Study of the Rural Development Project Final Report Tool number 36 (C_3) Description of respondents Component C: comparative sample: representatives of gmina authorities who had applied for co-financing under Component C, but were refused Operational base from which the sample was drawn Operational base: a list of gminas that had applied for support from Component C, but were not qualified; purposeful selection. Sample drawing method A 3-degree sample drawing scheme was applied. At the first stage, powiats were drawn in voivodship strata. The probability was proportional to the number of gminas in the powiat that had applied for co-financing, but were refused. At the second stage, gminas were drawn with the equal probability of selection. Finally, persons from gmina offices were selected with the focus on their ability to make statements concerning issues related to this part of the Project. Source: Consultant’s own analyses 56 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report 3 Analysis of Data Project evaluation has been based on empirical research work which provided quantitative and qualitative data. Qualitative data were obtained from questionnairebased interviews and from analyses of documentation. On the other hand, quantitative data were obtained from questionnaire-based interviews conducted on representative samples of beneficiaries described above. According to methodological rules of social sciences, the collected quantitative data have been subjected to statistical analysis, while qualitative data – to crosswise and longitudinal analysis in order to analyse specific and intersubjectivised features. The quantitative data and the qualitative data were treated as complementary, verifying and complementing one another. Due to so applied triangulation it was possible to obtain a complete picture of the evaluated Project. The following techniques were used for the analysis: • Analysis of the Project documentation contents; • Information from in-depth interviews was subjected to qualitative analysis, with the tactic of generating information meaning through: • o identification of patterns, main themes, repeating regularities, o grouping, i.e. matching objects and comparing them at the level of: events, actors, processes, places, circumstances, o noticing relations between variables, o building a logical chain of evidence. Statistical analysis of quantitative data, especially: o constructing indicators that reflect quantitative outcomes of the Project within the frames of four evaluation criteria; o projecting the delayed outcomes of the Project and their variability in time, o analysing dependence on structural variables (especially those characterising regions, where the Project was implemented) and on variables of a socio-demographic nature. On the basis of quantitative and qualitative data analysis, exhaustive and supported with empirical evidence answers to evaluation questions have been formulated. The process of evaluation was organised in a form of stage analysis. Particular stages were as follows: The first stage consisted in analysis of data (concerning Project issues and areas) from two basic sources. For the first, from a base consisting of all materials prepared during the evaluation survey and filed in Consultants' Database. These materials were distributed according to experts’ declarations of their information needs submitted before the survey began. For the second, these were secondary data from official publications of Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 57 Final Evaluation Study of the Rural Development Project Final Report the Main Statistical Office, Labour Offices, Local Data Banks and other covering the Project area. This analysis was used as a basis for the first part of evaluation effected by the top-down approach method. The first stage was closed with formulation of preliminary theses and expectations concerning Project evaluation within the scope which was possible at this stage of analysis and on the basis of available data. The top-down approach enabled the evaluators to identify the problems to which particular Components and Sub-Components of the Project responded. Moreover, the analysis provided a basis for evaluation of particular Components’ rationale and for identification of alternative forms of support that potentially could be more justifiable in the given area. Moreover, objectives and activities of particular Components and Sub-Components were evaluated at this stage, with a special focus on characteristics of interrelations between objectives of the entire Project and the logic of particular Components. Besides, the evaluators assessed the relevance of particular Components, i.e. the degree to which the Project objectives had identified target beneficiaries’ (persons, regions) real problems and needs and had responded to them. At this stage, relevance of selection of beneficiaries was evaluated, i.e. it was checked, whether the Project really addressed those beneficiaries who had originally been indicated as its target group. Assessment of objectives and activities of particular Components was closed with the evaluation of Project targeting, i.e. the degree to which the Project implemented the general assumptions of State policy in this field. The top-down analysis was closed with a diagnosis of the situation before and after specific Components, addressing in particular – where justifiable – economic indicators “before and after” the Project in the areas and groups covered by Components, in comparison with those, that had not been covered by the Project. Here, effectiveness of all Components was evaluated too and the assumed objectives were compared with those achieved and the utility aspect was assessed through confrontation of a given region’s and individual beneficiaries’ needs and problems with the real achievements of the Project after its completion. The second stage of work included a bottom-up analysis and it began with analysis of questionnaire data collected for particular Components and Sub-Components. These enabled the evaluators to assess Project net outcomes and outputs, on the basis of the most probable variant of events in the areas and groups covered by the survey, if the Project had not been implemented. At this stage, the focus was on identification of these elements of evaluation, that could not have been effected on the basis of secondary data, i.e. on evaluation of additional (external) effects of the Project, both positive and negative, and a “what – if” analysis. Thereafter – on the basis of Proeko documents again – those elements of the Project were evaluated that could be assessed on the basis of opinions (e.g. relevance, targeting, awareness, etc.) and such elements that could be assessed at a low deaggregation level mainly. At this stage of analysis it was possible to identify and measure costs of the Project and other outlays and as a result – to analyse the economic aspect of particular Components. Moreover, the financial leverage was assessed, i.e. it was indicated whether access to any other external funds was gained due to the Project. The next element of analysis included identification and measuring of Project net outputs and outcomes and its effectiveness, i.e. assessment, to what degree the planned outputs had really been produced. Analysis of efficiency was the third stage of work. With this end in view, Project costs and benefits were estimated and compared – the particular focus was on outlays incurred within the Project to create new off-farm jobs. This analysis was carried out in two phases. Firstly, opinions about Project outcomes were collected directly from entities participating in its implementation. Secondly, Project outcomes were compared with benchmarks of assistance or Government programs and – where possible – with market indicators (e.g. in the sphere of infrastructural projects). 58 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project Final Report Analysis of opinions provided by entities participating in Project implementation enabled the evaluators to identify the outcomes of the Project (including those non-economic ones). These data were obtained from entities involved in Project implementation at the operational level. The group of these entities included gminas, trainees, loan advisers, labour offices and others. Usually, these data were of a qualitative and subjectivised nature. The research tools of Project evaluation were the main source of these data. On the other hand, analysis of benchmarks enabled one to answer the question, whether the outcomes of the Project were much different from the outcomes of similar programs. The comparison of respective indicators enabled one to evaluate RDP in an objective manne Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 59 Final Evaluation Study of the Rural Development Project Final Report IV Outcomes Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 61 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. 1 Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 63 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. 1.1 Top-down analysis 1.1.1 Identification of Problems Component A Responds to The purpose of Component A was to support existing business entities and to assist in establishing new ones, as well as to support diversification of business activity in rural areas and small towns. The area of Component A covered five voivodships: KujawskoPomorskie, Małopolskie, Podkarpackie, Warmińsko-Mazurskie, Zachodniopomorskie. Component A adds to achievement of the Program’s overall objective: increased offfarming employment in rural areas. Problem characteristics According to the Program Appraisal Document, the existing and expected high level of uneployment in rural areas was the problem to be faced by Component A. In 1999, 2171 out of 2489 gminas were inhabited by rural population. The rural population accounted for 38.2 of Poland’s total population (14 759 thousand people). 27.1% of all the employed worked in agriculture (4 260 thousand people). This rate was much higher than in EU member countries. At the end of the 90-ies the uneployment in rural areas was a problem of primary importance. The group of registered unemployed had been growing rapidly in rural areas since 1998 and it amounted to 1 056 thousand in 1999 and 1 180 thousand in 2000. Figure 2. Unemployed inhabitants of rural areas in the years 1996-2000 1998-2000: total growth of unemployment by 41% 1 400 000 1 200 000 1 000 000 800 000 600 000 400 000 200 000 0 1996 1997 1998 Unemployed - total Women 1999 2000 Men Source: Statistical Yearbook, GUS, Warsaw 2001 Since the beginning of the 90-ies, the agriculture had been a buffer absorbing the surplus of labour force generated by the economy and it had a significant share in the hidden unemployment existing in rural areas6. It was estimated that at the end of the 90-ies the level of hidden unemployment exceeded 1 200 thousand people in Poland, about 900 6 See:. I. Frenkel, Ludność, zatrudnienie i bezrobocie na wsi. Dekada przemian, IRWiR PAN, Warszawa 2003. 64 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. thousand out of this number – in agriculture.7 As a result, the real rate of unemployment was much higher in rural areas than in cities. In the 90-ies the geographical diversity of the unemployment’s intensity increased. This resulted from the stronger fall of the rate of unemployment in the regions with the lowest rate in comparison with the regions with the highest rate. The best situation was observed in strongly urbanised voivodhips: Mazowieckie, Wielkopolskie and Małopolskie. The unemployment was highest in the areas where state farms had prevailed in the past, i.e. the north and the north-east of Poland. It should be stressed that despite significant differences in the unemployment level and the income level between regions, economic migration remains in trace quantities only. At the end of the 90-ies the following unfavourable features of the unemployment structure were taking root: • unemployment among young people, • high share of people with a low education level in the population of the unemployed, • high share of women in the total number of the unemployed, • long-lasting unemployment, • high unemployment among inhabitants of rural areas. The first four features listed above are characteristic for rural areas as well. These phenomena prove that at the end of the 90-ies the problem of high unemployment in rural areas was not only of the economic nature, but of social as well. In rural areas, the level of unemployment was adversely correlated with the inhabitants’ age. The unemployment hit young people worst – the rate of unemployment in the population of people aged up to 34 was higher than the national rate of unemployment. 7 The National Strategy for Growth of Employment and Human Resources Development for the Years 2000 – 2006, a document adopted by the Council of Ministers on 4.01.2000, p. 24. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 65 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Figure 3. Rate of unemployment in rural areas by the age of the unemployed as of the end of 1999 40,0% 35,0% 30,0% 25,0% average unemployment rate in rural areas 20,0% 15,0% 10,0% 5,0% 0,0% 15-19 20-24 25-29 30-34 35-44 45-54 55 and more Source: data published by GUS At the end of the 90-ies the unemployment was a very serious social and economic problem, but it should be stressed that a further growth of the number of the unemployed was expected. When RDP was being planned, the competitiveness of the Polish farming sector was very low on the background of other European countries. It was expected that in the years to come, prior to Poland’s accession to EU, restructuring activities would be effected in agriculture, to adjust the Polish country to the more intensive competition. The drain of labour force from farming was one of the forecasted consequences of the reforms that were planned. To sum it up, at the moment when Component A was planned and commenced, there was a serious problem of unemployment in the rural areas of Poland, the problem which was to deepen further. Causes of problems It is hard to indicate one cause of the high level of unemployment in rural areas – this was a complex problem at the end of the 90-ies and it remains such now. The following key causes can be named8: • collapse of state farms (PGR) that provided job to many inhabitants of the country; • strongly partitioned land property, high share of small farms; • relatively low level of farms’ technical equipment; • population’s low economic and social activity; a significant degree of acceptance of the situation; 8 See: T. Hunek (editor.), Rolnicza Polska wobec wyzwań współczesności, IRWiR PAN, Warszawa 2002. 66 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. • low mobility of rural dwellers; • low level of education and qualifications, often insufficient to take up any off-farm job (both in the form of hired labour or self-employment); • weakness of institutions and organisations supporting rural areas development, including local administration; • insufficient development of technical, social and cultural infrastructure which sets barriers to establishing new businesses; • low level of rural dwellers’ income restricting the demand for non-agriculture goods and services offered by new businesses. Component Rationale Evaluation Bearing in mind the scale of unemployment in rural areas, it seems that there it was impossible to solve the problem within the prospect of a few years. Due to overemployment in agriculture, locating off-farming businesses in rural areas was the main opportunity to absorb some of the unemployed there. The Common Farming Census of 2002 showed that farmers undertook off-farming activity in retailing and wholesale first of all (19.5% of farms running non-farming activity), in industrial processing (17.8%), building sector (5.1%), transport and storing (4.8%)9. The rural dwellers low mobility eliminated economic migration to cities in practice. Diversification of business activity in rural areas was determined by: • the number of investors, sector-related first of all, commencing or developing such activity • the number of people deciding to take up their own business activity. Activities of Component A focussed on stimulating the second of the above factors. Meanwhile, the economic situation in the country began worsening in 2000, bearing an adverse effect on activity of both sector investors and candidates for business people as well. Selection of Regions for Component Implementation When selecting regions to be covered by Component A, four basic criteria were used: (i) high unemployment rates in rural areas, (ii) high share of households with state agriculture as main source of income in 1988, (iii) high number of employed in state agriculture as a percentage of total population working in agriculture in 1988 and (iv) state-used land as a percentage of total agricultural land in 1988. We consider these criteria to be correct. Additionally, we have analysed the selected voivodships against the background of the national average, using supplementary indicators as of 1999. 9 A. Ciołkosz, Charakterystyka rolniczej przestrzeni produkcyjnej Polski, GUS Warszawa 2003. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 67 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Table 4. Comparison of voivodships selected for Component A with the national average, as of 1999 Item Voivodships 1 2 3 4 5 National average Share of rural dwellers in the total population 37.7% 49.6% 58.9% 39.9% 30.7% 38.2% Rate of unemployment 16.9% 10.2% 14.5% 22.4% 18.1% 13.1% GNP / 1 inhabitant (current prices, PLN) 14 121 14 231 11 685 12 341 15 924 15 914 Share of population with primary, incomplete primary education, or no education at all 41.2% 38.0% 42.2% 46.1% 40.2% 40.0% Share of people employed in agriculture, hunting and forestry 26.1% 34.2% 45.7% 24.7% 14.6% 27.5% 1 – Kujawsko-Pomorskie, 2 – Małopolskie, 3 – Podkarpackie, 4 – Warmińsko-Mazurskie, 5 – Zachodniopomorskie Source: Statistical Yearbook, GUS, Warsaw 2000, Statistical Yearbook for Voivodships, GUS, Warsaw 2001 Among the selected voivodships, least favourable values of indicators accepted for the analysis were found in Podkarpackie and Warmińsko-Mazurskie. In case of the remaining regions, the indicators are more positive, but still slightly lower than the national average in most cases. Additionally, taking into account that until the end of the 80-ies, state agriculture had been strongly developed in these voivodships, we consider the selection of regions for Component A as correct. 1.1.2 Evaluation of Component A Objectives and Activities Evaluation of Component Objectives According to the Operational Instruction adopted in 200010, the objective of Component A was to assist the Government in promoting micro-enterprises to create and maintain selfemployment in rural areas. The Component was expected to promote off-farming selfemployment in rural gminas of five voivodships. The objectives set for the Component added to achievement of one of three objectives of the Program: increasing off-farming employment. 10 68 Operational Instruction, Warsaw 9.03.2000, p. 14. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Project objectives Figure 4. Component A objectives and activities in the context of RDP objectives Comp. objective Increasing off-farming employment Supporting the process of strengthening self-government and regional development Assistance in building institutional potential necessary to obtain EU pre-accession and structural funds Component activities Assisting the Government in promoting SMEs to create and maintain self-employment in rural areas Micro-credits Subsidies Technical assistance and training of service providers Source: Consultant’s own analysis Considering the scale and the nature of the problem of unemployment in rural areas, it seems that the objective set for the Component is the right response to the problem. The objective covers one of the methods to activate off-farming activity in rural areas – establishing new micro-businesses. It is worth stressing that this involves twofold effect: • economic – through creation and development of off-farming enterprises; • social – through promoting activity and hence transforming the rural dwellers’ mentality. Evaluation of Component Activities Relevance in Terms of Secondary Data The Component objective achievement was supported by three groups of activities: • providing micro-credits amounting to USD 5000 (an equivalent in PLN); • providing one-time grants for purchasing equipment, up to PLN 3600; • providing advisory services and training to micro-entrepreneurs. The problem of financing (micro-credits and grants) There is no doubt that the limited access to business financing was one of the main barriers to establishment and development of small enterprises in Poland at the end of the 90-ies. This problem concerned rural areas in particular. Most often, any fund that were available (bank loans) involved the necessity to tide over complicated procedures and thereafter to meet rigorous formal requirements. Persons initiating business activity and small companies with no loan history faced biggest problems with obtaining any external financing.11 It should be added that in most cases the value of rural property 11 See: report titled Realizacja zaleceń Jednostki Zadaniowej ds. Uproszczenia Otoczenia Biznesu (BEST), Raport - Rzeczpospolita Polska, Warszawa 2001, p. 21. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 69 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Components was insufficient to serve as a collateral. Besides, financial institutions were not much interested in financing small entities, due to their significant fixed costs of small loans and credits maintenance. On the other hand, leasing services satisfied a part of micro-enterprises’ needs only. With the purpose of improving accessibility of external sources of financing to SMEs, collateral funds had been established since 1994. The National Fund for Loan Collateral of BGK (National Economy Bank – Bank Gospodarstwa Krajowego) had the broadest geographical range. Additionally, a dozen or more local funds and two regional funds operated in Poland in the year 2000. Local funds were established under the Regional Initiative Program, their capitals being financed from Phare funds to a high degree. The territorial coverage of local funds was usually limited to one or a few several gminas, often situated in rural areas. On the other hand, regional funds were set up as a result of the Polish-British Program for Supporting Entrepreneurship and were financed by the British government. 12 There was a number of micro-credit organisations in Poland at the end of the 90-ies, e.g. Fundusz Mikro, FDPA, FWW, Inicjatywa Mikro, Fundusze Rozwoju Przedsiębiorczości. But according to analyses carried out by the Foundation of Social and Economic Analyses Centre (Fundacja Centrum Analiz Społeczno Ekonomicznych) in 2001, the sector of loan organisations involved the total amount of PLN 200-250 million only in its activity13. One should take into consideration additionally that the major part if this amount – low, considering the needs – was distributed within urbanised areas. Since the beginning of the 90-ies, a project titled “Active Programs for the Labour Market Financed from the Labour Fund” has been implemented. Various instruments were used under this project, including loans for the unemployed to launch a business.14 Since 2000, the European Fund for Rural Development in Poland, Counterpart Fund commenced implementing a project titled “Micro-loans for Off-farming Business Activity in Rural Areas”. Young people and less wealthy inhabitants of rural areas in ten voivodships were target groups of the Program. This was a pilot project and it was implemented in cooperation with selected co-operative banks. 15 According to analyses carried out at the turn of the XX and the XXI centuries, more than 82% of small enterprises declared that they did not intend to use any non-commercial sources of financing.16 This resulted partly from the fact that relatively modest funds were involved considering the needs and that a number of barriers existed on the part of rural dwellers. The most significant of them included: • a barrier of lacking knowledge of financing sources; • a mental barrier – the reluctance to go into debts; • a barrier of geographical accessibility. 12 Polska Fundacja Promocji i Rozwoju Małych i Średnich Przedsiębiorstw: Finansowanie Rozwoju małych i średnich przedsiębiorstw, wrzesień-listopad 2000, p. 51 and further. 13 Editor: E. Balcerowicz, Mikroprzedsiębiorstwa – sytuacja ekonomiczna, finansowanie, właściciele, CASE, Warszawa 2002, p. 154. 14 Ibidem, p. 199. 15 Ibidem, p. 193. 16 Raport o stanie sektora małych i średnich przedsiębiorstw w Polsce w latach 1999-2000, PARP, Warszawa 2001, p. 272. 70 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Using the sources of financing referred to above was connected with a problem of knowing and recognising small entrepreneurship support organisations. According to the aforementioned research projects, more than 60% of entrepreneurs did not know any small business support organisation.17 The rural dwellers’ reluctance to go into debts (mental barrier) was a significant constraint to financing small business. During the survey carried out for the purpose of the present RDP appraisal, 69.7% of respondents who intended to launch their own business declared that they did not intend to use any loans or credits within a year. At the same time, 80.8% of them need funds to launch a business. The barrier of geographical accessibility resulted from the fact that units supporting beneficiaries had been located in larger places first of all. Mobile employees visiting beneficiaries at their places of residence were not often used. The SME sector’s difficulties with access to sources of financing were confirmed by a research carried out by the Ministry of Economy in the year 1999 on a national sample of small and medium-sized enterprises. The research showed that only 16% of SME-sector companies financed their activity with loans on a regular basis. This group included medium-sized entities first of all.18 On the other hand, analyses of the Foundation for Education and Scientific Research of the Marketing Analyses Centre Indicator carried out in the year 2000 showed, that small enterprises and micro-enterprises used bank loans seldom – only 25% of companies used loans – mainly in PLN and contracted occasionally. Most of the companies covered by the research relied on their own funds (90%) and family loans (35%). Such sources as leasing and other, e.g. loans from assistance funds (10% and 7% respectively) were used very seldom.19 To sum it up, there is no doubt that the access to financing is one of conditions of creating and developing business activity in rural areas. Since mid-90ies a number of instruments and institutions supporting people who launch their own business have emerged. It should be stressed however that these activities concentrated in urban areas to a high degree. We therefore consider the micro-crediting and granting activities under Component A as relevant to beneficiaries’ real needs. The problem of education (advisory services and training) Lacking knowledge of accountancy and tax regulations or how to run a business is one of important constraints to small entrepreneurship development in rural areas. As it has already been indicated earlier, this applies also to potential sources of financing. It should be stressed however that it is not information about sophisticated mechanisms of financing that potential beneficiaries lack, but the elementary knowledge from the field of economy and management. Rural dwellers’ low level of education was and still is one of the reasons behind this state of affairs. In the middle of the 90-ies, only 1.9% of rural areas population had a university background ( 9.8 % in cities). 17 Ibidem. Questionnaire survey of the Ministry of Economy: SME’s Economic Situation in the Opinion of Entrepreneurs” conducted in September 1998, June 1999, January 2000. 19 Survey of the Foundation for Education and Scientific Research of the Marketing Studies Centre – Indicator, September 2000. 18 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 71 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Figure 5. Education of the population of rural and urban areas in 1995 university post-secondary secondary general secodary technical and vocational primary incomplete 0,0% 10,0% 20,0% 30,0% 40,0% 50,0% rural urban Source: Statistical Yearbook, GUS, Warsaw 2000 In order to provide support to SME sector, including rural areas, in the field of advisory, training afnd information services, a network named Krajowy System Usług (National System of Services) was established in 1996. The network has comprised more than a hundred regional and local units supporting small and medium sized enterprises. 20 Under another initiative, by the year 2000, 61 Entrepreneurship Support Centres, 34 Entrepreneurship Development Funds and 31 Entrepreneurship Incubators had been set up in Poland21. These institutions’ tasks included providing training and advisory services to candidates for entrepreneurs and for small entrepreneurs operating on the market. There is no doubt that lack of knowledge in the field of economy and management is one of key barriers to development of entrepreneurship in rural areas. In the 90-ies activities intended to minimise this barriers were taken up, one should bear in mind however that the fight with the problem is of a long-term nature. In our opinion, the advisory and training activities proposed under the Component are adequate to beneficiaries’ needs. Additionally, these services constitute a good supplement to micro-crediting and granting activities. Evaluation of Component Activities Relevance in the Light of Survey of Potential Beneficiaries The maximum amount available under a micro-loan and a grant covered the needs of a major part of beneficiaries. Financial needs of more than a half of respondents remained within the limits of amount offered under Component A (5 thousand dollars). 59.6% of respondents indicated amounts within the limit of PLN 25 thousand. It is worth noting at the same time however that some of the beneficiaries had bigger financial needs. 20 Polska Fundacja Promocji i Rozwoju Małych i Średnich Przedsiębiorstw: Finansowanie rozwoju małych i średnich przedsiębiorstw, wrzesień-listopad 2000, p. 14. 21 Ibidem, p. 8. 72 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Figure 6. Financial needs of loan program potential beneficiaries up to PLN 5 thou. 8,9% more than PLN 5 thou., up to PLN 10 thou. 23,8% 26,8% more than PLN 10 thou., up to PLN 25 thou. 19,6% more than PLN 25 thou., up to PLN 50 thou. 11,1% more than PLN 50 thou., up to PLN 100 thou. 5,1% more than PLN 100 thou. 4,7% do not know 0,0% 10,0% 20,0% 30,0% Source: consultant’s own analyses As the analysis of the project beneficiaries’ needs shows, their needs differ only to a minimum degree only from the structure of loans contracted by average micro-companies on the banking market, where a demand for loans up to PLN 25 thousand prevails too. Figure 7. Value of loans contracted by micro-enterprises 56% up to PLN 25 thou. 29% PLN 26 – 50 thou. 9% PLN 51 – 100 thou. more than PLN 100 thou. 0% 6% 10% 20% 30% 40% 50% 60% Source: BPS report „FIRMY MIKRO 2004” Namely such a structure of needs for financing results from the structure of microenterprises’ investments first of all. According to data published by GUS for the year 2003, as much as some 42% of companies making investment outlays (about 13% of all micro-enterprises) made outlays amounting to PLN 5000 thousand, while only 7% made outlays that exceeded PLN 100 thousand. Hence, the limits set on loans granted under Component A, where more than a half of credits allowed remained within the limit up to PLN 12000, should be regarded as pertinent. The potential beneficiaries were expecting a very low rate of interest – as much as 79.1% of respondents indicated 3% as the maximum acceptable interest. However, one may presume on the basis of RDP outcomes and on the basis of information from the commercial banking market on the other hand, that some of these persons would be able to accept a higher interest rate. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 73 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. During the survey, the potential beneficiaries indicated aspects of obtaining a micro-loan that were relevant from their point of view. Figure 8. Potential beneficiaries’ rating of the relevance of selected aspects of obtaining a micro-loan atractive rate of interest loan disbursement timing prompt analysis of documents flexible terms of repayment free of charge assistance in applying little amount of documents required high micro-loan/ grant amount facility of filling the application 0,0% 20,0% 40,0% 60,0% 80,0% 100,0% very important important Source: Consultant’s own analyses The analysis enables one to conclude that such aspects as: attractive interest rate, timing of the loan disbursement and prompt analysis of documents are most important for the beneficiaries. The above factors were given 80% of indications as very important each. Evaluation of Component Activities Relevance in Terms of Beneficiaries Survey Beneficiaries formulated very positive opinions about the Component. The utility of financing in the form of micro-loans and grants combined with advisory services was given positive marks by 98.7% of respondents, 73.7% out of this group giving a very good mark. 74 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Figure 9. Utility of the form of financing applied under Component A 100,0% 90,0% 80,0% 70,0% 60,0% 50,0% 40,0% 30,0% 20,0% 10,0% 0,0% 1 graduate very good good 2 unemployed rather good 3 farmer rather bad 4 self-employed business person bad very bad Source: Consultant’s own analyses Opinions on the form of financing do not differ much in particular groups of beneficiaries. The very good reception of the Component is a product of Component marks: • micro-loan amount – 82.7% of positive marks, 28.0% out of them being very good; • grant amount – 66.3% of positive marks, 24.0% out of them being very good; • interest on micro-loans – 93.0% of positive marks, 40.7% out of them being very good; • facility of filling the application – 93.0% of positive marks, 39.3% out of them being very good; • facility of collecting the required documents – 91.0% of positive marks, 37.3% out of them being very good; • time of documents analysis – 86.0% of positive marks, 31.7% out of them being very good; • time from the positive decision to disbursement – 86.3% of positive marks, 37.3% out of them being very good; • creditor’s flexibility towards terms of repayment – 88.0% of positive marks, 33.3% out of them being very good. It should be noted that the group of very positively received factors included also those of a key importance for the potential beneficiaries, i.e. attractive interest rate, payment timing and prompt analysis of documents. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 75 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. During in-depth interviews, the beneficiaries and loan advisers stated that the services provided under the Component were sufficient. Very few respondents referred to any problems encountered when obtaining a micro-loan or a grant (10.7% of respondents). The group of main problems included: the need to collect many documents (4.3% of respondents), long time of response to application (4.0%), too little funds to be obtained (2.7%). Few persons came for additional consultations to a loan adviser, except for assistance in filling and submitting the applications (21.3% of the respondents). 97.0% of the respondents gave a positive answer to a question if they would participate in a similar RDP Loan Program in the future and this fact provides evidence that the Component fitted beneficiaries’ needs well. Participation Evaluation Component A was addressed to inhabitants of rural areas planning to launch an offfarming business and to existing micro-enterprises employing up to 5 people. The Component area covered rural gminas, rural-urban gminas and towns with a population of up to 20 thousand. Initially, the Program embraced three social groups (farmers running farms up to 10 hectares, micro-entrepreneurs, registered unemployed). In March 2002 a decision was made to extend the Project on graduates of middle schools and secondary schools. Figure 10. Beneficiaries of grants and micro-credits Graduates Farmers Micro-entrepreneurs Registered unemployed 0 500 Grant beneficiaries 1000 1500 2000 2500 3000 Micro-credit beneficiaries Source: Final report of RDP A - " Micro-loans ", BGK Criteria used for selecting beneficiaries enabled access to the Component to a wide group of people and this should be regarded as positive. The fact that farmers running small farms were included in the Project is conducive to concentration of farms and this is an advantageous tendency, considering how broken up the farming land is. The fact that people who had already been running their business were invited to the Project made it possible to maintain the existing entrepreneurship, but on the other hand it might cause that as a result of the Component, the existing jobs would be protected and no new jobs would be created. The fact of extending the target group of the Project on graduates gives evidence to appreciation of the problem of young people’s unemployment in rural areas. 76 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. We are estimating the number of potential beneficiaries at some 1 million people in 5 voivodships covered by Component A (data for the year 1999): • the unemployed – about 0.4 million people, • graduates of secondary and post-secondary schools – about 0.05 million people 22, • farmers running farms up to 10 hectare – about 0.5 million people, • micro-entrepreneurs – 0.04 million people 23. The group of Component A beneficiaries amounted to 4655 people24, with the following breakdown into regions: • Małopolskie – 1060 people; • Zachodniopomorskie – 610 people; • Warmińsko-Mazurskie – 853 people; • Kujawsko-Pomorskie – 976 people; • Podkarpackie – 1156 people, i.e. less than 1% of the target group. This means that the scale of the Project was insufficient to solve the problem of unemployment. Nevertheless, according to the RDP authors’ intentions, the Component was supposed to be a pilot project and it was not its objective to solve the problem of unemployment in rural areas. It is not surprising that most beneficiaries of the Component were from the smallest, but the most resilient group of potential beneficiaries – micro-entrepreneurs. About 6% of potential credit receivers – micro-enterprises operating in the area covered by the Project – participated in the Project. In practice, no school graduates joined the Project – less than 1.0% of potential beneficiaries. Component Beneficiaries The fact that people commencing their business activity prevailed among beneficiaries of micro-loans should be regarded as very positive. This means that activities of the Component contributed to creating new enterprises first of all, instead of to supporting the existing ones. 22 The number of graduates in rural areas and in cities up to 20 thou. inhabitants is not available. According to statistics published by GUS, the total number of secondary, technical, vocational and post-secondary schools graduates amounted to 179.2 thousand in 1999 on the territory of five regions covered by Component A. It has been assumed for the purpose of the present study that about 50 thousand persons from this group lived in areas covered by the Component. 23 Available data quoting the number of micro-enterprises are not broken down into cities and rural areas. Experts agree that the vast majority of business entities from this group is localised in strongly urbanised areas. (see:. ed. E. Balcerowicz, Mikroprzedsiębiorstwa – sytuacja ekonomiczna, finansowanie, właściciele, CASE, Warszawa 2002, p. 51). It has been assumed for the purpose of the present report, that 10% of all micro-enterprises are situated in rural areas. 24 It was possible for a beneficiary to be granted more than one micro-credit under the Project, but no situation like this occurred. Consequently, the number of beneficiaries equals the number of micro-credits that were allowed (4655). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 77 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Figure 11. Number of micro-loan beneficiaries in a breakdown by gender and years of running a business activity 2000 1500 1000 500 0 Women Men Entrepreneurs starting business activity (including those running a business not longer than 12 months) Entrepreneurs continuing their activity Source: Final Report of RDP A - " Micro-loans ", BGK Such a high participation of people commencing their business activity in the Component resulted from additional preference given to this group – a non-repayable grant. Women constituted a significant part of the beneficiaries (43%). We consider this to be a very positive fact, regarding the high share of women in the structure of unemployed in rural areas. Figure 12. Share of women in the structure of beneficiaries of selected loan projects 80% 61% 60% 48% 43% 42% 40% RDP Fundusz mikro (Poland) Inicjatywa mikro (Poland) 40% 20% 0% 10 selected micro-loan institutions 12 micro-loan institutions in Eastern European countries Source: base of DORADCA Consultants Ltd. As the diagram presented above shows, the proportion of women in RDP is minimally higher than in other similar projects implemented in Poland and intended to support micro-businesses. On the other hand, the higher share of women in foreign projects results from the fact that supporting women’s entrepreneurship is often the main objective of these projects. 78 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. The high unemployment of young people is one of important problems in rural areas. Component activities reached this group of people only partly. The group of people aged up to 25 had a very small representation in the Component (14.8%). The participation of the next age group was much more significant – persons aged from 25 to 35 accounted for 32.4% of the beneficiaries. Other age groups had much lower shares in the Component. It is not surprising that the group of people aged above 55 was the least numerous (4.8 %). Figure 13. Age of micro-loan beneficiaries 1000 800 600 400 200 0 up to 25 yrs 25 - 35 yrs 36 - 45 yrs Women 46 - 55 yrs above 55 yrs Men Source: Final Report of RDP A - " Micro-loans ", BGK It is comforting that most of the beneficiaries were from rural areas, i.e. from places where: • it was more difficult to get an off-farm job, • it is more difficult to obtain financing for launching a business than in cities. Table 5. Number of micro-loan and grant beneficiaries Place of residence of loan and one-time grant beneficiaries Mikro-loans Grants Town up to 20 thous. inhabitants 1692 851 Countryside 2963 2093 Source: Final Report of RDP A - "Micro-loans", BGK Targeting Evaluation Component A fitted well the State’s policy in the field of rural areas which had been expressed in the following government documents: • Mid-Term Strategy for Agriculture and Rural Areas Development (1998); • Treaty for Agriculture and Rural Areas (1999); Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 79 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. • Cohesive Structural Policy for Rural Areas and Agriculture Development (1999).25 The Component objective was consistent with the Government’s efforts to reduce the unemployment in rural areas and to diversify sources of rural dwellers’ income by the means of developing off-farm jobs. 1.1.3 Situation prior to and after Implementation of Component A Effectiveness Evaluation Achievement of Component and Program Objectives According to the assumptions, the objective of Component A was to support the Government’s activities in promoting small enterprises to create and maintain selfemployment opportunities in rural areas. The objective was expected to lead to achievement of one of three objectives of the whole Program: increasing off-farming employment in rural areas. For the purpose of monitoring implementation of the Component, output indicators had been set together with the expected performance indicators in the Program Appraisal Document. All of the indicators that had been planned were exceeded. The number of micro-loans and grants allowed exceeded the plan by 72% and 96% respectively. The group of advisory services beneficiaries was larger by 133% than planned. It seems that such good quantitative results were achieved due to creating 6 times more micro-loan provider outlets than planned, these outlets having additionally the nature of mobile loan advisers. Table 6. Component A objectives – plan and performance Output indicators Plan Performance Engagement of at least one service provider 1 2 Establishment of at least 4 service provider outlets in the rural areas of the selected voivodships 4 24 Disbursement of 2,700 micro-loans to previously unemployed and micro rural entrepreneurs 2700 4655 Disbursement of 1,500 grants to eligible beneficiaries 1500 2944 Advisory services delivered to at least 2,000 beneficiaries 2000 4655 Service Provider Subsidy Dependence Index that has decreased every quarter and with a value less than 80% after two years of operations for each participating service provider <80% FWW – 16,1% FDPA – 45,9% Source: PAD, p. 43, 54, Final Report of RDP A - "Micro-loans", BGK Activities performed under the Component were expected to contribute to creating new off-farming jobs. At the stage of applying, the beneficiaries quoted the number of new jobs in their applications – both those resulting from self-employment and from employing hired personnel as well. According to these declarations, 4788 jobs should have been created. 25 “Cohesive Structural Policy for Rural Areas and Agriculture Development” accepted by the Council of Ministers in 1999 indicates the necessity of incresing off-farming employment in rural areas. 80 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. After the loan allowing stage was closed, loan agents verified data quoted in the applications in the period from May to October 2004. The real number of new jobs created under Component A amounted to 2941. Figure 14. Comparison of instruments used under Component A with the number of jobs created Number of jobs created (verified data) Nu,ber of jobs created (data from applications) Number of grants Number of micro-loans 0 1000 2000 3000 4000 5000 6000 Source: Final Report of RDP A - "Micro-loans", BGK The number of declared jobs exceeded the number of micro-loans to a minimum (4788 against 4655), but the number of jobs that were created in reality accounted for 61% of those declared. This means that one micro-loan generated 0.63 of a job. It is worth noting that the number of jobs was close to the number of grants (2941 against 2944). Questionnaire surveys of beneficiaries carried out within the present evaluation show that since March 2005 only 7.3% did not survive. This means that in some cases of microloans the declared employment did not take place or employees were dismissed after some time. One can not preclude a situation where some beneficiaries declared a too optimistic employment level to enhance their chance for being allowed a micro-loan. Some doubts can be noted as far as definition of a new job is concerned. This is connected with participation of persons launching a business in the Component. According to the terminology accepted for the purpose of RDP, a new entrepreneur is a person who launches a business as a result of participation in the Component or who launched an activity like this not earlier than 12 months prior to participation in the Component. This means that some of micro-enterprises that have been classified as established as a result of the Component, were set up to one year earlier, irrespective of participation in RDP. The same comment applies to some of the new jobs. Supporting the process of building the institutional capacity necessary to obtain EU preaccession support and structural funds was one of the three overall objectives of RDP. The interviews conducted with representatives of implementing institutions – the Rural Development Foundation and the Foundation for the Development of Polish Agriculture – show that participation in RDP enabled both institutions to increase their competence in large projects management. But participation in the Project did not result in any considerable know-how extension in the field of micro-credits. Both organisations had already had much relevant experience before RDP was launched. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 81 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. To sum it up, in our opinion the objective of Component A has been achieved. All of the key performance indicators have been exceeded. At the same time, the activities contributed to achievement of Program general objectives. Micro-loans and grants combined with advisory services resulted in increasing the off-farming employment in rural areas by nearly 3 thousand jobs. To a lesser degree did the activities support building the institutional potential necessary to obtain EU funds (the third general objective of RPD) – through development of Component performers. Reservation: the present paragraph does not include any comparison of voivodships participating in Component A with remaining regions in order to identify the Component effects. This has not been taken up due to the too small scale of Component A effect, which is too difficult or impossible to capture. 1.2 Bottom-up Analysis 1.2.1 Program Costs and other Outlays Identification and Measuring The cost of Component A implementation comprised direct costs and additional costs incurred by the beneficiaries. Direct costs The budget allocated to Component A was one of the smallest items in the entire RDP budget (about 6% of the whole budget) and it was higher from the budget of SubComponent B3 only. The costs of Component A implementation turned out to be lower than assumed. The difference between the performance and the plan resulted from – among other factors – changes in USD rate of exchange. The costs had been planned with the USD rate of exchange of PLN 4.27. Meanwhile, when activities were being performed, the rate of exchange went down much (the average rate in 2003: PLN 3.89 zł, the average rate in 2004 – PLN 3.65). Table 7. Budget of Component A Plan thous. USD Item Micro-loans Grants Administration costs, payment to performers Total training, Plan PLN thous. Performance PLN thous. Performance / Plan % 13 890 59 352 56 061 94% 2 900 12 726 10 593 83% 2 260 9 284 8 007 86% 19 050 81 362 74 661 92% Source: PAD, p. 55, Monitoring table as of 31.03.2005, MIS base. The share of administration costs, training and payment to performers accounted for 11.0% of the total budget value. An analysis of cost per single loan / grant shows about 300 USD. This level is comparable with operating costs of a loan provided to a microbusiness by commercial banks using traditional approach to credit capacity analysis. Bearing in mind that under Component A the cost covers a wider area of activities (including training), it should be stated as a fact that the efficiency of work under 82 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Component A was higher than the efficiency of commercial banks offering loans to microcompanies. About two/thirds of funds for implementation of activities analysed here were provided by the World Bank. The plan and the performance of financing particular activities of the Component were as follows: • World Bank – 75% of the amount of micro-loans, 100% of administration costs, training, payments to performers; • Providers of services (FWW, FDPA) – 25% of the micro-loans amount; • The Polish Government – 100% of the grants amount. Table 8. Parties financing activities of Component A Item Performance PLN thous. Share in the total amount % World Bank 50 053 67% Polish Government 10 593 19% Service providers 14 015 14% Total 74 661 100% Source: MIS base, Interim Report on Program Management no 17, PCU 2005 According to the latest data obtained from PCU directly (17.05.2005), the degree of the World Bank’s loan absorption reached 99.9% of the planned level. Additional costs 26.3% of the surveyed beneficiaries indicated additional costs related to receipt of microloans and grants. Firs of all such items were listed as: costs of travelling (73.4%), additional dedicated time of work (39.2%). The beneficiaries were not able to estimate the value of additional costs. Project organisation 47.5 persons were involved directly in work under Component A, including 24 loan advisers. The table below shows selected efficiency indicators of Component A organisation against selected benchmarks. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 83 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Table 9. Selected efficiency indicators of Component A organisation against benchmarks Benchmark Selected 110 microcrediting institutions 12 micro-crediting institutions in Eastern European countries RDP Loan receivers / personnel (quantity) 128 90 98,0 Loan receivers / loan officers (quantity) 308 171 194 (%) 44,7 54.8 50,2 Loan officers / employees Source: Consultant’s own analyses The table above enables one to conclude that the administrational efficiency of the Program represented a level which was to a minimum better that the average of European micro-loan funds. Considering organisation of crediting micro-enterprises in Polish banks, one may confirm that Component A staffing was correct in terms of quantities. 1.2.2 Program Outputs and Outcomes Identification and Measurement Outputs Generated by the Component 4655 micro-loans and 2944 grants amounting to the total of PLN 66 659.8 thousand were allowed under Component A (micro-loans – PLN 56 061.4 thousand, grants – PLN 10 598.4 thousand). 5123 loan applications and 3045 grant applications were submitted. The very low share of rejected applications – 9.1 % in case of micro-loans and 3.3 % in case of grants – may suggest that the criteria used for allowing them were too liberal. Nevertheless, this thesis is challenged by the low share of overdue loans (4.91%). One should note additionally that only one loan was considered to be lost. The number of training participants was equal to the number of micro-loan beneficiaries. 84 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Figure 15. Quantities of micro-loans and grants allowed by voivodships 1400 1200 1000 800 600 400 200 0 Podkarpackie WarmińskoMazurskie KujawskoPomorskie quantity of micro-loans Zachodniopom. Małopolskie quantity of grants Source: Final Report of RDP A - "Micro-loans", BGK A half of micro-loans and grants was allowed in the area of two voivodships: Podkarpackie and Małopolskie. Fewest micro-credits and grants were allowed in Zachodniopomorskie voivodship– only 12.0% of the total number of micro-credits and grants. The average value of a micro-loan amounted to PLN 12 555. Micro-credits ranging from PLN 6001 to 12000 prevailed (29.6%). It should be noted however that the values of micro-loans were very much differentiated. Table 10. Quantity of micro-loans by loan amounts Micro-loan amount Quantity of micro-credits allowed up to PLN 6000 PLN 6001 12000 PLN 12001 18000 More than PLN 18000 1085 1377 1287 906 Source: Final Report of RDP A - "Micro-loans", BGK A definite majority of grants was disbursed in the maximum amount of PLN 3600. In some isolated cases the amounts paid were minimally higher. Micro-loans and grants were supposed to be spent on running micro-businesses in various off-farming sectors. Trade and services were most popular among the beneficiaries (73.% of all micro-loans in total). Some persons launched or developed agritourist activity (17.1%) Few of the beneficiaries launched or developed off-farming production (9.9%), which can be associated with the fact that this activity is more capitalintensive. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 85 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Figure 16. Types of off-farming contracted for business activities micro-loans were 1200 1000 800 600 400 200 0 Production Services Business launched Agri-tourism Trade Business continued Source: Final Report of RDP A - "Micro-loans", BGK Persons launching a business accounted for nearly two/thirds of the beneficiaries (65% of micro-loans) – this was most visible in agri-tourism and least – in manufacturing. Figure 17. Types of business activity – RDP against the structure of microenterprises and against other micro-loan programs 100% 80% Production 60% Service 40% Trade 20% 0% Structure of Polish microenterprises RDP Micro Fund Micro Initiative Source: Final Report of RDP A - "Micro-loans", BGK, base of DORADCA Consultants Ltd. An analysis of the Component A beneficiaries by sectors indicates that this structure is close to the structure of micro-enterprises activity all over Poland. It is worth stressing that the share of beneficiaries running service activities is over-proportional against the background of the structure of all micro-enterprises and the structure of beneficiaries of other assistance programs (e.g. Micro Fund and Micro Initiative). Considering sectoral structures of Western European countries and USA, namely services have the highest growth potential in the Polish economy. Funds obtained as micro-loans and spent on investment developments were most often assigned for machines, devices and tools (30.8% of the respondents) and for a car (20.5%). Besides, furniture was purchased relatively often (17.9%), renovation and building works were carried out (16.4%). Grants were mainly spent on machines, devices and tools (20.8% of the respondents), office equipment (19.0%) and furniture (19.0%). 86 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. For the sake of comparison, the structure of outlays made by micro-companies last year is presented below. Figure 18. Structure of investment outlays made by micro-companies in 2004 4% other developments * purchase of complete technological lines 1% building new premises, plants 5% purchase of means of transport 9% purchase of single machines and devices 11% purchase of computer software 12% purchase of computer hardware 13% current renovations and rebuilding 18% 0% 5% 10% 15% 20% Source: BPS „Firmy Mikro 2004” Funds from micro-loans assigned for purchasing current assets were mainly spent on goods (46.7% of respondents) and building materials (26.7%). Long-term micro-loans prevailed, this resulting mainly from the fact that monthly repayment instalments were lower in this case. More than 75% of micro-loans were contracted for a period longer than 2 years. Table 11. Quantity of micro-loans by the pay-back periods Pay-back period (months) Quantity of micro-loans allowed Do 6 7-12 13-24 25-36 9 288 861 3497 Source: Final Report on RDP A - "Micro-loans", BGK The average interest rate on a micro-loan was 6.26%, it should be noted however that in case of 51% of micro-loans, the interest rate did not exceed 5.00%. Meanwhile, the cost of commercial loans amounted from some eight, nine to a dozen or more percent. The micro-loans allowed were of a high credit quality – few of them were overdue or lost; the repayment indicator exceeded 91% in all categories. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 87 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. The share of overdue loans accounts for 4.91% now and the share of bad debts is 0.02%. When comparing the quality of RDP portfolio with the situation on the Polish banking market (more than ten percent of bad debts) and with other loan institutions (the average bad debt index for analysed institutions in the world is about 2.3% and in European countries 3.9% respectively), one should stress that loan risk management was very effective under RDP. Table 12. Micro-loans repayment rate Repayment period in months Micro-loan amount (in PLN) Up to 6 from 7 to 12 from 13 to 24 from 25 to 36 Up to 6000 100,0% 98,8% 96,2% 95,2% 6001-12000 100,0% 100,0% 91,9% 96,4% 12001-18000 100,0% 100,0% 92,1% 97,5% More than 18000 100,0% 100,0% 93,7% 94,3% Source: Final Report on RDP A - "Micro-loans", BGK Specific outcomes The beneficiaries, when asked about the effects of participation in the Component, indicated the possibility of launching a business first of all and this is not surprising considering the figures quoted earlier. In case of already existing entities, the assistance enabled them to increase their sales and in consequence – profits. Not many enterprises expanded the geographical coverage of their activity or implemented new technologies due to micro-loans and grants. This can be associated with the fact that funds available under the Component were relatively small. In some exceptional cases the assistance saved businesses from liquidation of bankruptcy. Figure 19. Outcomes of participation in Component A as seen by the surveyed beneficiaries Business saved from liquidation / bankruptcy New technologies implemented Territorial expansion 5,3% 7,7% 8,0% New products launched 19,0% Employment increased 19,7% Profit increased 26,3% Sales increased 30,7% Business launched 0,0% 50,7% 20,0% 40,0% 60,0% 80,0% 100,0% Source: Consultant’s own analyses 88 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Not more than one/fifth of the beneficiaries increased the employment in their companies as a result of being allowed a micro-loan and a grant. Most of them employed one (60.4% of the beneficiaries increasing the employment) or two persons (26.4%). In few cases only, the employment was increased by three or more persons (13.2%). As far as the group of persons increasing the employment is concerned, men prevailed (67.8%), as well as persons running a business before the Program commencement (62.7%), trade businesses (43.8%), situated in mixed urban-rural gminas (59.3%). Figure 20. yes, negative changes occurred 2% Changes in the financial, social or professional standing resulting from participation in Component A, as seen by the surveyed beneficiaries do not know, hard to say 6% nothing has changed 25% yes, positive changes occurred 67% Source: Consultant’s own analyses Most of the respondents regarded the effect of the Component on their financial, social or professional situation as positive. Most positive answers were recorded in WarmińskoMazurskie (80.7% of beneficiaries from this region), least – in Zachodniopomorskie – 55.6% respectively. The highest level of satisfaction was indicated by school graduates (92.9% of positive answers), the lowest – by entrepreneurs (65.8% respectively). The main objective of the Component had been to minimise the unemployment in rural areas, but this does not mean that maintaining the existing employment in microcompanies was a less important effect of the Program. Unfortunately, there is no data available to identify the number of people who lost their job in cases when they did not participate in the Component. 1.3 Summary 1.3.1 Component A Cost / Benefit Analysis Benefits Quantifiable benefits achieved as a result of Component A activities include first of all outcomes in the form of new jobs. These outcomes translate into economic benefits: reduced amounts of unemployment benefits being paid, increased Budget income from income taxes and turnover taxes. The amount of new jobs declared by the beneficiaries in their applications amounted had been 4788. Following verification, the real number of new jobs amounted to 2941 (61.4% Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 89 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. of what had been declared). The new jobs were taken up by various groups of beneficiaries. Figure 21. Structure of new jobs created as a result of Component A activities, as declared by the beneficiaries Others 18% Unemployed 38% Entrepreneurs 5% Graduates 2% Farmers 37% Source: Final Report of RDP A - "Micro-loans", BGK The appraisal of benefits resulting from creating jobs varies in particular groups. It is most advantageous to employ the unemployed or school graduates as potential unemployed. It is less beneficial to employ persons who are not provided for by the unemployed support system and/or generate tax income for the State budget (the phenomenon of personnel transfer within the existing jobs). Table 13. Comparison of budget benefits sources when creating new jobs for various groups of beneficiaries Source of benefits for the State Budget Unemployed, graduates (40% of beneficiaries) Farmers, other groups (60% of beneficiaries) Costs of State’s support to an unemployed √ - Fiscal benefits resulting from an unemployed person to take up a job √ √ Source: Consultant’s own analyses For the purpose of the Program, the annual cost for the Budget to support one unemployed was calculated at PLN 2700 in 1997, while potential fiscal benefits at an amount ranging from PLN 1,100 to 3,30026. The total financial benefits for the budget resulting from unemployment reduction by one person ranged from PLN 3,800 to PLN 6,000 in the year 1997. A new calculation of the above values has been made for the purpose of this evaluation, on the basis of data of the year 2003 (see: details in paragraph 7.1 – Analysis of State Budget’s financial benefits resulting from creation of new jobs). The annual cost of supporting one unemployed person amounted to PLN 1,300, while fiscal benefits ranged from PLN 3,300 to PLN 10,500. Thus, the total financial benefits for the budget resulting from reduction of unemployment by one person ranged from PLN 4,600 to PLN 11,900 in the year 2003. 26 90 PAD, p. 25-26. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. We have estimated that so calculated annual benefits resulting from creation of new jobs under Component A ranged from PLN 11.2 million to 32,5 milion, this amount consisting of: • the group of unemployed and graduates (1176 people) – ranging from PLN 5,400 thousand to PLN 14,000 thousand (with reduction of costs of supporting the unemployed and tax benefits taken into account) • other persons (1765 people) – ranging from PLN 5,800 thousand to PLN 18,500 thousand (with tax benefits taken into account) Most probably, the level of income of persons taking up newly-created jobs was below the average national salary, therefore the estimation of benefits at the lower limit of the amount quoted above (PLN 11.2 million) is more realistic. Moreover, the Component activities resulted in a number of less measurable effects that are hard to convert into zloties. The following benefits were indicated most frequently in interviews made with the Component beneficiaries and performers: • rural dwellers’ increased confidence in assistance funds; • micro-loans made popular among entrepreneurs and potential entrepreneurs. For example, loan advisers stressed that many beneficiaries learned about the Component from persons who had already benefited from RDP. Questionnaire surveys confirm this thesis – 44% of the respondents learned about the Components from other beneficiaries or from acquaintances. Interview respondents indicated also that the number of potential beneficiaries reporting was increasing as the Component implementation was proceeding. Although the Component was closed last year, such persons happen to come even today. The thesis about rural dwellers’ increased confidence in assistance funds can be supported by the fact that 90% of the surveyed beneficiaries stated that they would participate in a similar loan project in the future, should this be possible. Costs The group of costs incurred to gain the benefits quoted above includes27: • costs of grants: PLN 10.6 million; • costs of administration, training, payments to performers: PLN 8.0 million; • indirect costs incurred by the beneficiaries (costs of travelling, additional dedicated time of work) are hard to estimate. To sum it up, the total cost of Component A implementation amounted to some PLN 18.6 million. 27 The item of costs does not include micro-loans that were repayable. With regard to the credit portfolio very high quality, we do not add the forecasted value of lost micro-loans to costs either. To make the account simpler, the difference between commercial rate of interest on loans and the real interest on micro-loans has been disregarded. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 91 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Comparison of costs to benefits The cost of PLN 18.6 million made it possible to gain annual benefits of about PLN 11.2 million. Such investment’s payback period is very short – it is about one year and a half. According to the Monitoring Table, the cost of creating one job amounted to PLN 10,169 (around USD 3,000) and this value was much lower than estimations accepted in the Project Appraisal Document (around USD 5,000)28. Nevertheless, comparing net effects of the given project in absolute values against programs from other countries is made difficult due to differences in both the environment in which the programs function (GNP per capita, income levels, economy competitiveness, rate of unemployment, etc.) and objectives, scopes and target beneficiaries of these programs. This thesis can be supported with the following examples: • Grant Fund Component under PHARE-STRUDER PL9207 Program – implemented in the years from 1993 to 1999 – the cost of 1 declared job amounted to some EUR 3.5 thousand. • According to Business Development Funds (USA) program guidelines, the grants allowed in the amount of USD 11.8 million are expected to result in saving / creating about 39 000 jobs and about 1000 enterprises, meaning that the cost of co-financing per one employed is around USD 303 and per one company – USD 11.8 thousand. • In the 90-ies MBLP was implemented in Zimbabwe. The project involved matching micro-companies with large receivers on the one hand and assistance to small companies to adapt to large receivers’ requirements on the other hand. As a result, about 110 new enterprises were established and after few years their employment amounted to some 1000 employees. This employment was achieved at a relatively low level of outlays – around USD 1350 per one enterprise, i.e. USD 150 per one person. 1.3.2 Long-term Impact of Component A It is very difficult to identify the long-term impact of Component A, as not much time has passed since the activities were completed – the last micro-loans were allowed in September 2004. It is worth stressing however that the questionnaire survey of Component beneficiaries shows that beneficiaries ceased business activity in 7.4% of cases only. This is a highly satisfactory outcome, particularly when compared with the assumed indicator of enterprises survival rate. According to the Project Appraisal Document, the Component activities would be regarded as cost-effective if more than 59% of micro-businesses survived after two years from launching.29 1.3.3 Component A Activities Implementation of Component A commenced with signing a contract for Component implementation with the National Economy Bank (BGK) in April 2001. BGK signed also 28 29 92 PAD, p. 23. PAD, p. 23. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. an agreement with the VPIU. Thereafter, a procedure of selecting consultants – technical assistance providers was initiated. In April 2001 some delays occurred due to problems with selecting service providers whose tasks were expected to include: micro-loans and grants disbursement, providing training and advisory services. Absence of foreign subjects’ interest in providing such services (whose participation was initially required by the World Bank) was one of the causes of delays. Hence, the process of selecting a provider of services was protracted. Eventually, two subjects participated in the tender procedure. When financial proposals were opened in February 2002, it turned out that they were much above the budgets envisaged for the task. Thus, it was necessary to run additional negotiations to reduce service providers’ costs and this caused another delay in Component implementation. The service contract was signed as late as in October 2002. The first applications for micro-loans were submitted in December 2002. Disbursement of micro-loans began in January 2003 and of grants – in February 2003. Figure 22. Basic tasks of Component A – implementation timing Source: Consultant’s own analyses based on Interim Reports of Program Management, PCU The contract for technical assistance and training activities was signed with Facet BV – a consulting company – in December 2001. Technical assistance, which initially consisted in developing and implementing an evaluation and monitoring system, began in January 2002. In March 2002 Facet BV conducted training on micro-loans for implementing institutions. The same month, a decision was made to include graduates of middle schools (gimnazja) and secondary schools in the Program. The next training conducted by Facet BV was dedicated to operating the micro-lending process monitoring system – PROMES. In May 2002, the eligibility criteria were changed (inhabitants of cities up to 20 thousand and owners of farms up to 10 hectares were admitted to apply for micro-loans). In the III and the IV quarter of 2002 Facet BV was continuing training activities and conducted two study visits to Bosnia-Herzegovina and to Montenegro, where a highdeveloped market of micro-financing services exists. These visits were attended by representatives of service providers, BGK and MoEL. To provide a maximum effectiveness of micro-loans allowed under the Component, micro-loan advisers were hired. The process of recruitment was finalised in March 2003. They were also trained by Facet BV. The process of allowing micro-loans and grants ended in September 2004 and a summary conference was organised in the same month. The Foundations keep monitoring loan beneficiaries through loan advisers. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 93 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Beneficiaries’ Assessment of Providers’ Activities When responding the questionnaire survey, the beneficiaries were asked to assess the work of loan advisers. The respondents gave very high marks to their advisers – only 4% of beneficiaries gave them negative marks. Figure 23. Beneficiaries’ Assessment of loan advisers’ work Very bad 1% Rather bad 2% Bad Rather good 1% 5% Good 28% Very good 63% Source: Consultant’s own analyses The resultant average mark was composed of very good marks for advisers’ knowledge, time efficiency, availability, commitment to client’s problems, individual approach to the client, politeness. Advisers’ politeness was given highest marks (70% of very good marks), time efficiency – lowest (59% of very good marks). Evaluation of Activities 1. The phase of Component start-up was much protracted – the first micro-loans were disbursed more than a year and a half after the contract with BGK was signed. 2. At the beginning of Component implementation, significant, several months’ delays in disbursing funds to beneficiaries occurred. In the service providers’ opinion, some beneficiaries resigned from the micro-loan allowed to them. 3. The sequence of activities was correct in general, concerns being raised by the fact that recruitment of loan advisers was commenced after the process of collecting applications for micro-loans and grants began. 4. In our opinion, study visits to Bosnia-Herzegovina and Montenegro were valuable. 5. The Component implementation was supported by a significant amount of training activities and these, in combination with the study visits results, strengthened the effectiveness of goals achievement. 6. We are of a very high opinion about the work of loan advisers. Synergy of Components The construction of the Rural Development Program allowed for potential synergy between Component A and Sub-Component B1. It was assumed that some persons 94 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. trained under Sub-Component B1 would wish to commence their own business activity, using micro-loans and grants. But during the Project, circumstances occurred that made it difficult to generate the effect of synergy. In particular, this applies to the very significant delay in implementation of Component A and to the fact that there was no closer co-operation between the subjects implementing both Components. On the other hand, some of the activities increased the chance of synergism, e.g. training for loan advisers provided by IMC, where forms of support available under SubComponent B1 were presented and also the fact that a company granting micro-credits was entrusted with the task of running the business assistance centre in Zachodniopomorskie voivodship. In some few cases, we encountered information that business assistance centres co-operated with loan advisers. Figures give evidence to the scale of synergy. Databases of both Components show that 433 persons participated in Component A and in Sub-Component B1. The services of business assistance centres was expected to be a source of synergism in SubComponent B1 – 20,172 persons used these services during RDP. These participants were from 5 voivodships where Component A was implemented and from two additional regions: Lubelskie and Świętokrzyskie voivodships. To conclude it, it seems that the level of synergy was relatively low (several percent). 1.3.4 Conclusions and Recommendations for the Future Conclusions The Component objectives and activities were adequate to the social and economic situation in rural areas during the Program implementation. We assess positively the choice of voivodships where the Component activities were implemented. Quantitative achievements of the Component can be considered as satisfactory when compared with the expected outcomes. 2941 new jobs were created as a result of Component activities. As far as the group of persons increasing the employment is concerned, men prevailed, as well as persons running a business before the Program commencement, trade businesses, situated in mixed urban-rural gminas. Component A gave preference to newly launched micro-businesses, supplementing micro-loans with non-repayable grants. The Component activities made it easier for beginners to launch a business, as it was much simpler to obtain funds and at a lower cost. The attempt to reach one of potential beneficiary groups – school graduates turned out unsuccessful. In general, young people participated in Component A to a low degree. Respondents to survey questionnaires assessed the utility of financing in form of microloans and grants combined with advisory services very well. Very few persons indicated any problems encountered during participation in the Component. The work of loan advisers from foundations implementing the Component activities were assessed very high. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 95 Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. The effect of the Component at the regional and voivodship level was limited (participation of 1% of potential beneficiaries) due to relatively small funds. At the beginning of Component implementation, entities from the micro-lending sector were relatively well developed in Poland, therefore the Component implementation contributed to improvement of the existing mechanisms only and not to any qualitative change of micro-loan programs. The first stage of Component implementation (selection of service providers) was definitely too long – more than a year and a half. At the beginning of the micro-loan disbursement process serious delays in the flow of funds to beneficiaries took place. The financial mechanisms were made more efficient as a result of corrective actions. We assess positively the friendliness of micro-loan and grant procedures. Applications for financial support were relatively easy to fill. Only the section dedicated to financial plan of the business was difficult for the beneficiaries, but on the other hand it made them think over the practicability of their ideas thoroughly. As a result, the risk involved in microbusinesses being launched was reduced. The risk management of the Component was highly satisfactory – the share of lost loans is close to zero. There are no proofs of synergy effect between Component A and Sub-Component B1. The comparison of benefits generated by the Component with its costs is highly satisfactory – we estimate the payback period of the investment at some three years. Recommendations for the Future Mobile and active loan advisers formed an effective channel of beneficiaries procurement which proved efficient in RDP and which is worth using in further micro-loan projects. It is worth running promotional and informational activities of further loan programs in cooperation with local authorities and with institutions enjoying population’s confidence, e.g. farming advisory centres. This increases the range of information campaign and results in increased trust of potential beneficiaries to the program. This especially important that some beneficiaries were distrustful of the Component. To reach school graduates as a target group of loan programs one needs to develop entirely new tools, different from those used under Component A. It is worth considering increasing the limit of micro-loan amount to be provided under future programs. Both the beneficiaries and the providers of services indicate that the assistance was insufficient in some cases. This applied in particular to persons developing their existing businesses. It seems that increase of the loan amounts should be accompanied by extended maturity term, so that this does not involve a significant growth of the debt monthly costs. We suggest the Component beneficiaries’ standing be surveyed in two-three years – after the maturity term of micro-loans expires – in order to identify the long-term impact of Component A. We recommend that when planning further micro-loan programs, solutions offered to various groups of beneficiaries are differentiated. 96 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component A Walery Marcinowicz, Piotr Wróbel, Ph. D. Existing micro-businesses As it has been mentioned before, existing micro-businesses were Component’s most active beneficiaries. This means that in further programs satisfactory effects will be achieved if a similar functionality of the program is adopted. It would be advisable however to consider making the program more commercial in nature – e.g. introducing sanctions in the form of requirement to return the difference between the interest rate applied for the program and the interest rate of the commercial market in cases where a beneficiary ceases implementing the program objectives (e.g. does not employ new personnel at the time declared) or delays loan repayment; dispersal of grants by the means of reducing the preferential level of subsidising for individual beneficiaries. Newly launched microbusinesses We suggest an entirely different approach to persons who consider launching new businesses (the unemployed, farmers and school graduates). Considering their fear of risk involved in setting up an own company and their lack of experience in running a business on the other hand, it is necessary to provide “leadership” and support to these people throughout the complete cycle of establishing a company. Program activities would begin with a help to the most enterprising potential entrepreneurs in developing an idea (e.g. by the means of sharing the knowledge about successes of other microbusinesses, providing information about areas of potential activity with the best prospects for development in townships and villages) and would include a full support in the period of commencing and running the activity at the initial stage. Such programs would work as “ tailored services for SME” and they should be based on those specific sectors that develop most intensively in townships and villages of Poland or on those that experienced their prime in other countries that joined EU in previous decades. It would be possible to identify key success factors for selected sectors and thereafter – to support the “newly-baked” businessmen in the whole process of development: from talking potential businessmen into taking the risk of launching a company, through providing specialist training legal support, assistance in finding contractors, help with the accountancy (e.g. duty hours of an accountant, co-financing of bookkeeping), consultations in the field of quality issues concerning the given sector, technological support (e.g. providing access to the Internet, facilitating preferential purchase of computers / arranging instalments for a payment, reference visits in Poland and abroad, manuals for particular sectors), up to providing support in gaining the capital needed for further development. Agri-tourism is the first area that could be covered by a program like this. Cost/benefit effective implementation of a micro-loan program requires an appropriate scale (number of beneficiaries). It seems that Component A can become a good reference point in this respect when designing next micro-loan programs. It seems that it would be worth working out a continuation of the Component based on the potential beneficiaries’ increased confidence in such tools and using the tools and mechanisms of micro-loans distribution developed under the Project. In our opinion, next programs are desirable – first of all with regard to the high proportion of effects (reduced unemployment) to the outlays made to implement the program. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 97 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 2 Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 99 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 2.1 Top-down Analysis 2.1.1 Identification of Problems Sub-Component B1 Responds to In the second half of the 90-ies rural areas were covering more than 90% of Poland’s territory and rural population accounted for nearly 40% of the total population of the country. Persons dealing with agriculture accounted for some two-thirds of the rural population, persons owning no arable land being the rest. In the analysed period, labour-related problems of the farming population (having a relationship with the farming activity) included first of all low productivity of work when compared with other economy sectors and with farming sectors of other European countries. In the year 2000, the employment in agriculture reached 28% of the total employment, while this sector’s share in GNP accounted for 3% only30, which means that productivity of the farming sector was at the level of about 1/11 of the average national labour productivity in Poland31. This caused that the level of most farmers’ income dropped to some 60% of the income gained by the remaining social and economic groups in the second half of the 90-ies (in 1989, before the transformation, it accounted for 115% of other groups’ income)32. The worsening of the agricultural sector’s standing in the 90-ies resulted in a rapid fall of agriculture’s share in GNP, but without any significant changes in farming employment, therefore, with the absence of alternative employment directions in rural areas this translated into unemployment growth. Immediately after economic reforms, the agriculture played the role of a buffer, absorbing the surplus of the labour force generated in the economy. This resulted in a disproportionate hidden unemployment, particularly in regions with many small farms that absorbed more unemployment than regions dominated by large commodity farms. Nevertheless, most of the farming population did not qualify for unemployment benefits, because they owned land and many others did not wish to apply for social assistance, this leading them to poverty. The demographic and social structure of the unemployed was rather unfavourable in rural areas, mainly in terms of education. The unemployed from rural areas are characterised by a significantly lower share of persons with university degrees (this share did not exceed 3%) and with secondary education (less than 30%) than the unemployed in cities and at the same time, the share of persons with vocational and primary educational background is higher in rural areas (more than 70% in total). Migration, so intensive in the 80-ies, contributed to the outflow of the best educated part of this population to cities and although these movements slowed down in the 90-ies, the imbalance still exists in the sphere of education33. Many authors indicate that in case of rural population, the low level of qualifications is one of substantial barriers to job seeking (in terms of both hired labour and self-employment as well) in off-farming sectors34. In spite of a significant improvement of the rural 30 Rocznik Statystyczny GUS (Statistical Yearbook), Warszawa 2002. Productivity is defined as a proportion of performance to outlays, therefore the approximate productivity of the farming sector is 10.7% (3%/ 28%*100). 32 Analiza produkcyjno-ekonomicznej sytuacji rolnictwa i gospodarki żywnościowej w 2002 roku, IERiGŻ, Warszawa 2003. 33 W trosce o pracę. Raport o rozwoju społecznym. Polska 2004, UNDP, Warszawa 2004. 34 Inter alia: Chaplin, H. i Milczarek, D. Dywersyfikacja działalności ekonomicznej gospodarstw rolnych w Polsce. Wyniki badań w województwie podlaskim, podkarpackim i dolnośląskim, in: Wieś 31 100 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. population’s education level in the last decade35, the level of human capital which is lower than in cities, reduces off-farm employment opportunities. Labour Market Trends and Prospects in Rural Areas of Regions Covered by SubComponent B1 and Outside before Program Implementation Inclusion of Sub-Component B1 to the RDP as an element of the “human capital development” was intended as a response to needs of the rural population whose economic situation was difficult. Sub-Component B1 was planned as an assistance in adjusting and improving this group’s professional qualifications and skills, as well as in using them in a fuller manner. The response to these needs was supposed to be consistent with the economic development possibilities and with the situation on local labour markets. Since the very beginning, particular focus was on supporting entrepreneurship, increasing income levels, reducing the sphere of poverty in rural areas and promoting less developed regions. Hence, selection of voivodships for the Program, as well as allocation of funds to individual regions were based on such criteria as: (i) rural population in the voivodship,(ii) GNP per capita and (iii) unemployment rate36. To assess the accumulation of barriers to occupational activation, trends within the basic economic and social variables in the 7 selected voivodships before the Program implementation have been analysed and compared with characteristics of regions that did not participate in the Program. In our opinion, the problems of a deepening nature require intervention in particular. Since 1997, the proportion of employed inhabitants of rural areas had been falling significantly. It had grown in Mazowieckie and Wielkopolskie regions only (by 1% and 5% respectively). Although in the years 1996-1999 the number of working people was relatively higher among inhabitants of rural areas, the drop of this number was more serious in regions covered by Sub-Component B1 (dynamics of fall higher by nearly 1 pp.) than in regions not covered by the Program. i Rolnictwo 4/2003 and Wilkin, J., ed. Podstawy strategii zintegrowanego rozwoju rolnictwa i obszarów wiejskich w Polsce, WNE UW, Warszawa 2003. 35 According to data of the Common Farming Census (Raport z wyników Powszechnego Spisu Rolnego 2002, GUS Warszawa 2003). 36 Project Appraisal Document, Warsaw 2000. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 101 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table 14. Employed in total, inhabitants of rural areas in the years 1996-1999 Voivodship 1996-1999 1999/1996 mean* % change 1996 1997 1998 1999 Dolnośląskie 108090 108333 105306 98757 105049 -9% Kujawsko-pomorskie 80383 81557 80806 76141 79693 -5% Lubelskie 97068 98804 94108 88396 94510 -9% Lubuskie 40619 41854 41071 38652 40532 -5% Łódzkie 97072 101124 100955 93369 98078 -4% Małopolskie 112925 117054 116577 109507 113975 -3% Mazowieckie 153649 160047 162193 155949 157924 1% Opolskie 57586 57534 55604 51786 55576 -10% Podkarpackie 94421 98602 98340 92724 95988 -2% Podlaskie 36907 37782 37283 34290 36540 -7% Pomorskie 72619 79382 78256 71623 75394 -1% Śląskie 126426 130110 127769 116692 125142 -8% Świętokrzyskie 56164 56502 55167 51040 54674 -9% Warmińsko-mazurskie 56325 59248 58404 52469 56550 -7% Wielkopolskie 166501 174071 179620 174594 173633 5% Zachodniopomorskie 60824 65556 64835 58008 62229 -5% B1 voivodships 76972 79691 78403 72586 76866 -5,7% Other voivodships 72475 74788 74039 69021 72546 -4,8% Difference in basis points (B1 – other voivodships) -0,9% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS There were more inhabitants of rural areas employed in the industrial sector in regions that did not participate in B2. The number of rural dwellers working in this sector grew, but the growth was definitely bigger in voivodships remaining outside B1 (1.9% against 0.4%). Thus, both the absolute and the relative difference in the number of people working in the industrial sector deepened to the disadvantage of B1 regions. The absolute number of persons working in the farming sector was slightly bigger in voivodships outside B1, but they were more numerous at the same time (9), therefore the dynamics of employed rural dwellers from rural areas are more important and these were higher in the voivodships participating in B1 than in the rest of the country. This may be an indicator of both positive and negative changes. The first would mean that the agricultural sector is subject to more rapid restructuring in the regions covered by B1 and that rural population begins working in the industrial and service sectors. This variant is of little probability however, as no considerable growth of employment in industry was observed. Thus, the faster dynamics indicate rather a transfer to non-earned forms of maintenance (disability pensions, retirement pensions, allowances and benefits) and this thesis is probable, considering the fact that the farming population is growing older. 102 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. The relatively higher number of businesses registered in rural areas of voivodships covered by B1 when compared with the remaining regions is a positive phenomenon. But the growth of dynamics, although considerable (34.4%), was lower than in the voivodships remaining outside B1 (by 1.2 pp). Table 15. Inhabitants of rural areas working in the industrial sector in the years 1996-1999 Voivodship 1996-1999 1999/1996 mean* % change 1996 1997 1998 1999 Dolnośląskie 53580 52036 51042 48072 51142 -10% Kujawsko-pomorskie 29983 30669 30863 30700 30552 2% Lubelskie 31951 34023 31775 29738 31836 -7% Lubuskie 15523 16591 17092 16760 16481 8% Łódzkie 50089 52902 54426 50302 51898 0% Małopolskie 43504 44906 46092 42511 44232 -2% Mazowieckie 60058 64139 67543 60445 62974 1% Opolskie 20643 21130 21253 20045 20762 -3% Podkarpackie 33879 37349 37123 34761 35747 3% Podlaskie 10322 11103 11813 10519 10924 2% Pomorskie 28408 31382 32472 30297 30603 7% Śląskie 67655 70757 68706 60688 66841 -10% Świętokrzyskie 22141 22955 22511 21125 22173 -5% Warmińsko-mazurskie 17677 17848 20208 19185 18701 9% Wielkopolskie 68889 74721 81497 80627 76261 17% Zachodniopomorskie 21097 22456 22752 21852 22030 4% B1 voivodships 27422 28716 29028 27518 28162 0,4% Other voivodships. 33361 35484 36525 34004 34822 1,9% Difference in basis points (B1 – other voivodships) -1,6% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 103 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table 16. Inhabitants of rural areas working in the farming sector in the years 1996-1999 Voivodship 1996-1999 1999/1996 mean* % change 1996 1997 1998 1999 Dolnośląskie 13523 13558 11899 10244 12227 -24% Kujawsko-pomorskie 13296 12985 12120 10423 12152 -22% Lubelskie 7544 7815 6884 5751 6951 -24% Lubuskie 6652 6420 5914 4833 5911 -27% Łódzkie 6139 5942 5345 4595 5471 -25% Małopolskie 5269 5034 4549 3791 4625 -28% Mazowieckie 10089 10139 8784 8529 9356 -15% Opolskie 12936 12780 10954 9598 11482 -26% Podkarpackie 6526 6257 5731 5125 5885 -21% Podlaskie 3640 3745 3327 2956 3403 -19% Pomorskie 10830 11857 10792 9241 10638 -15% Śląskie 6753 6793 6045 4925 6079 -27% Świętokrzyskie 3412 3239 2789 2457 2950 -28% Warmińsko-mazurskie 12564 12809 11250 8682 11197 -31% Wielkopolskie 33093 32843 29495 26002 30216 -21% Zachodniopomorskie 14169 15389 13807 11201 13551 -21% B1 voivodships 7999 7969 7136 5995 7226 -25,1% Other voivodships. 9736 9938 8877 7690 9015 -21,0% Difference in basis points (B1 – other voivodships) -4,0% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS 104 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table 17. Entities registered in REGON system – private sector in rural areas in the years 1996-1999 Voivodship 1996-1999 1999/1996 mean* % change 1996 1997 1998 1999 Dolnośląskie 31881 36302 40445 43452 37765 36% Kujawsko-pomorskie 23850 28673 31884 34484 29447 45% Lubelskie 27568 31352 35196 37064 32586 34% Lubuskie 14597 16340 17505 18108 16582 24% Łódzkie 30339 34993 38647 40938 36000 35% Małopolskie 60234 63651 68060 71656 65758 19% Mazowieckie 68586 73253 82401 88339 77765 29% Opolskie 13854 15631 17688 19690 16572 42% Podkarpackie 33914 37601 42159 44993 39437 33% Podlaskie 12189 13702 15426 17125 14493 40% Pomorskie 23304 27049 30209 32974 28150 41% Śląskie 44765 48066 53640 56867 50615 27% Świętokrzyskie 18230 19333 22443 24240 20925 33% Warmińsko-mazurskie 14470 17500 19219 19697 17596 36% Wielkopolskie 51218 58364 65865 69922 60913 37% Zachodniopomorskie 20307 23647 26747 28967 24697 43% B1 voivodships 25594 28991 32359 34399 30146 34,4% Other voivodships 23656 26560 29659 32069 27804 35,6% Difference in basis points (B1 – other voivodships) -1,2% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS The growth of unemployment was a vital economic and social problem all over Poland in the second half of the 90-ies. It was experienced by voivodships from both groups, but there were considerable differences in this phenomenon’s intensity and dynamics. In the years 1998-1999 the absolute number of the unemployed living in rural areas of voivodships covered by B1 was by almost a half bigger than outside these regions (although there are more voivodships there) (Table A1 in Annex 1). Moreover, unemployed graduates constitute a bigger percent in B1 regions. Besides, the rate of unemployment and the proportion of the unemployed aged up to 25 are higher by as much as around 2.5 pp. At the same time, there were less job offers there at this time and much more unemployed per 1 job offer (e.g. in 1999 – 615 in B1 voivodships against 421 on the remaining territory). The fact that these tendencies tended to deepen was also disturbing37. 37 Trzeba zauważyć jednak, że i wśród województw nie objętych B1 są takie, gdzie rozmiary zjawiska nasilającego się bezrobocia były nawet większe niż w niektórych województwach objętych B1 (na przykład w województwie łódzkim czy dolnośląskim). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 105 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. In the years 1995-1999 employment indexes fell in B1 voivodships more than in the remaining regions (the difference in dynamics reached 2.2 pp.). The balance of internal and foreign migrations was also higher in B1 regions in these years, although it dropped in 2000 against 1995. (Table A2 and A3 in Annex 1). To sum it up, one may state that most of negative phenomena on the labour market in rural areas and all over Poland were of a deepening nature in the years 1996-2000. An analysis of GUS (Main Statistical Office) data shows that in the year 2000, before implementation of Sub-Component B1, the situation of regions selected for this implementation was much worse than in the remaining voivodships (Table A4 in Annex 1). For the first, GNP per capita was lower in voivodships covered by B1 and much lower than the national average – it was as low as 81.4% of the national average). For the second, B1 voivodships’ share of agriculture and rural areas was much higher in terms of the national product (4.8% in the added value against the national average accounting for 3.8) and in terms of farming employment as well (as much as 35% share of those working in the agriculture against 23.9% outside B1 – according to the Main Statistical Office, on the basis of data provided by labour offices). In general, regions participating in B1 were inhabited by rural population almost in a half (rural dwellers accounted for 46.3 of the total population), while this proportion was around one-third (35.4%) in the remaining voivodships. Although in the year 2000 the overall rate of unemployment was higher in the voivodships not participating in B1, the formal unemployment among the population having a relationship with a farm was higher in the regions covered by B1 (12.7% against 11.4%). In case of population having no relationship with any farm, the unemployed was the same, i.e. 27%. Moreover, there were much more small individual farms up to 3 hectares in the structure of farms situated in B1 area (potentially qualifying for support provided under this Sub-Component), but on the other hand the percentage of farms running an off-farming activity was considerably lower (by 1 pp.). Thus, any improvement of the situation on the labour market in regions covered by B1 was very much desirable, but the analysis of individual voivodships shows that it was possible to apply a slightly different selection of voivodships that would qualify for B1. A simple comparison of all voivodships according to the 3 most important criteria that have been used (i.e. GNP per capita, the proportion of rural population and the unemployment in rural areas) indicates that Małopolskie, Kujawsko-Pomorskie and ZachodnioPomorskie should have been replaced with Podlaskie, Opolskie and Łódzkie. On the other hand, if one uses cases studies rather than average values, it can be proved that the selected voivodships had some sub-regions with a particular accumulation of economic and social problems (e.g. regions of former state agriculture, etc.), this factor justifying their acceptance to the Program. Sub-Component B1 Rationale Evaluation As we have proved above, unemployment-related problems became visibly more intensive in the second half of the 90-ies and no market mechanisms that might lead to any solutions could be seen. According to estimations, the chance for creating new jobs begins with the economic growth at the level of some 5% (in cities first), while the economic growth was at the average level of 3.1% in the years 2000-200438. Moreover, the protracted economic transformation of the country and the generally weakened development of the world economy worsened the situation in industry and agriculture. In these circumstances, one could not expect that the private sector would engage in pro- 38 Rocznik Statystyczny 2004, GUS Warszawa and PG TOP Polska Gospodarka. Tendencje, Oceny, Prognozy, CASE, Warszawa 2005. 106 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. employment solutions, especially that the cost of labour was growing due to State’s growing fiscalism. As one can see, there was no practical chance for market-based solution of problems related to high unemployment in rural areas in this period, especially that a very intense demographic growth of labour resources was observed then. Since intervention was a necessity, one may ask whether commercial or public funds should have been involved. According to the Project Appraisal Document39, the variant of financing the Program with commercial funds had been taken into consideration. It was rejected however, due to the limited of appropriate instruments (such as municipal bonds) and due to concurrent limitations for financing public works, particularly in rural areas. One should agree that this was a serious barrier to using commercial funds. Thus, intervention with the participation of state funds was justified. Finding out whether the Sub-Component responded to the complexity of employmentrelated problems was another important question that needed to be answered in order to evaluate Sub-Component B1 rationale. As it is known, it was difficult to develop programs with a comprehensive approach to complex problems (e.g. that would combine educational solutions with self-governmental, health-related, infrastructural and other solutions) within the frames of State policy (and it still remains difficult, despite the fact that decentralisation of public finances has already been initiated). This results mainly from the sector approach prevalence. Each ministry seeks funds for its own activities, this leading to competition for funds instead to efforts to consolidate funds for specific purposes. Previous programs addressing rural areas, such as for example World Bank’s projects, ASAL (Agricultural Sector Adjustment Loan, 1994-1997) or Rural Environmental Protection Areas confirmed this tendency, as they concentrated on mono-sectoral solutions (e.g. on rural infrastructure or environment protection). This time, participation of the World Bank was supposed to facilitate the multi-sectoral construction of the Program and to fill the gap in long-term financing40. Thus, at the level of planning, the construction of Sub-Component B1, as well as the entire RDP construction was rather innovative and valuable, as it enabled one to depart from previous program models and to apply a program based on comprehensive approach to problem solving. One should consider it positive that at the stage of RDP planning, not only infrastructure and education, but human capital development was taken into consideration as well, i.e. the program of reorientation and redeployment (Sub-Component B1). Expected to respond to problems of rural population’s employment and entrepreneurship. One should stress however, that a program consisting of many Sub-Components may have an economic advantage over separate programs in such circumstances only, when the total of its Sub-Components’ outcomes is higher than the total of separate programs would be, in other words, when synergy effects occur. This problem will be discussed further in this evaluation study. Economic and Social Background of Sub-Component B1 As the analysis presented above shows, the problems of employment and human capital development in rural areas were complex: they were of an economic and a social nature. Thus, when evaluating Sub-Component B1 utility it is worth analysing, how these aspects had been perceived at the stage of constructing this Sub-Component dedicated to redeployment and reorientation. 39 40 PAD, Warsaw 2000. As above. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 107 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. According to the Project Appraisal Document, the basic economic intention behind implementation of the Sub-Component was to reduce budget expenses and to increase GDP as a result of transfer of employees from the sector where they are not productive and where they are paid public subsidies (such as farming), to such sectors where they can be more effective and where they will not be paid any public subsidies (industry and services). One should note at the same time that the GNP level will be growing only when retrained persons are employed in newly launched businesses or in workplaces with a higher productivity. This logic is definitely justifiable, one should remember however that if persons trained under the Program cause an effect of a definite push-out of persons employed previously on the same positions, the important social problem of structural unemployment will not be solved despite the improvement of economic efficiency (resulting from improved productivity). The objective of the Program was to provide comprehensive solutions to problems and not to “replace” them. Thus, our evaluation focuses on net effects of creating new jobs. As far as the social aspect of Sub-Component B1 background is concerned, it is based on the so-called “Social Appraisal”, i.e. case studies conducted for the purpose of the Program, consultations with specialists and reports on employment41. According to these studies and analyses, despite rural dwellers’ numerous declarations of their readiness to seek off-farm employment, the existing vocational training curricula do not guarantee that skills adequate to new economy’s needs are gained. Persons wishing to launch an own business expressed their frustration caused by the insufficient condition of rural infrastructure. The poor communication system made it impossible for rural dwellers to obtain information about employment opportunities and for enterprises – to maintain their contacts with the outlet. High costs of transport and little frequency of public transportation rides discouraged those inhabitants of rural areas who had thought of seeking off-farm employment and it limited country families’ financial capacity to payment for education of their children. Besides, the analyses have shown that women lose the fight for jobs as a rule and that there are much less women managing business entities. Lack of baby-sitting support and limited chances for finding a job are important factors behind this situation. All these social aspects seem to characterise the cause and effect situation and we shall find out in the further part of this evaluation, whether they were reflected inn the Program construction. To conclude, the rationale behind the project focussed on reorientation and labour redeployment – as Sub-Component B1 was – has to be assessed as very satisfactory, particularly when it constitutes a part of a comprehensive program (such as RDP) addressed to rural population. 2.1.2 Evaluation of Sub-Component B1 Objectives and Activities The main objective of Sub-Component B1 is to provide support to the process of economic development, to increase incomes and to reduce the sphere of poverty through providing assistance to inhabitants in the area of enhancing their human capital and improving its use42. The specific objective of Sub-Component B1 is to provide assistance economically disadvantaged and unemployed in rural areas to expand and use their human capital in 41 42 108 PAD, Warsaw 2000. RDP Operational Manual, Warsaw 2000, p. 45. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. response to economic and labour market opportunities43. Thus, the following two objectives can be identified: 1. Improvement and better use of qualifications (human capital), 2. Assistance in taking advantage of opportunities emerging on the labour market. Various materials44 present these objectives in a breakdown into three categories: 1. Assistance in improving one’s qualifications, 2. Assistance in finding a job on local labour markets, 3. Assistance in human capital adjustment to requirements of local labour markets. The objectives defined above give evidence to the fact that the Program was focussing on active forms of counteracting unemployment and that it was oriented towards providing help to the rural economically disadvantaged population and the unemployed. Two basic tasks were financed under Sub-Component B1: (i) economic and labour market surveys in the field of economy and labour market-related issues and (ii) labour redeployment services based on regional programs of the selected regions. The redeployment services absorbed 96% of the entire Sub-Component’s funds (USD 43.15 million out of USD 44.84 million). 45 The economic and labour market surveys were carried out in all voivodships covered by Sub-Component B1. Their objective was first of all to: provide short-term and long-term projections on economic trends and labour markets; develop co-operation and interrelations between Labour Offices and employers, support the process of planning redeployment services and develop skills and knowledge of Labour Offices’ representatives in the field of collecting data, carrying our research and forecasting. Thus, they constituted an important element of preparing and implementing the main part of the Program, i.e. redeployment services. According to PAD46, services financed under Sub-Component B1 were focussing on: (i) improving labour mobility and addressing frictional unemployment; (ii) enhance the quality of human resources via retraining – to address structural unemployment and improve productivity and (iii) supporting individuals to help them start and maintain microbusinesses and to communities to implement temporary community employment schemes and design local development plans – to address the lack of demand for labour47. The RDP Operational Manual defines six types of services48: 1. training services, 2. specialised employment services (advisory services), 43 Realizacja Podkomponentu Reorientacji/Przekwalifikowań. Aneks 3, Ministerstwo Pracy i Polityki Społecznej, Warszawa 2000, p. 275. 44 „Rural Development Program”, Information brochure issued by FAPA, Warsaw 2000. 45 Project Appraisal Document, Warsaw 2000. 46 Ibidem, p.23. 47 Ibidem. 48 “RDP Operational Manual”, Warsawa 2000. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 109 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 3. temporary employment services, 4. small business assistance services (assistance in launching and running a business), 5. small business incubators, 6. local development planning. Each of the service types seems to address at least one of the main objectives. Objective 1 is mainly achieved by the means of training services and temporary employment services. Objective 2 is mainly achieved by the means of professional advisory services, small business assistance services, business incubators and local development planning. Sub-component Activities Sub-component Objectives Program Objectives Figure 24. Sub-Component B1 objectives and activities in the context of RDP objectives Increasing off-farming employment Supporting the process of strengthening self-government and regional development Increasing incomes and reducing the sphere of poverty through assistance to inhabitants to improve and use their human capital better Employment services Temporary employment programs Business Assistance Centres Assistance in building institutional potential necessary to obtain EU pre-accession and structural funds Providing support to the process of economic development Training services Small business incubators Local development plans Source: Consultant’s own analyses It also seems that some services of Sub-Component B1 help achieve objectives of other Components. For example, business assistance centres and business incubators will also address one of Component A objectives: “Creating and Maintaining SelfEmployment of Rural Population”. The local development planning service may also address the Sub-Component B3 objective (“Increase effectiveness and efficiency of administration units in absorbing investment outlays”) and one of the Component A objectives (“Assistance to government institutions in promoting small enterprises”). The part of this study following below analyses the degree to which the redeployment program objectives responded to problems and needs of Program target areas and persons; the degree to which program objectives were interrelated with the State policy general guidelines for employment and human potential development and finally – what these services’ coverage was, i.e. how many people participated in the Program in comparison with the number of those in need. Sub-Component B1 Relevance Evaluation To find out, whether the objectives and the activities of Sub-Component B1 specified in the Operational Manual were relevant to the development-related needs of the selected voivodships, SWOT analyses included in regional strategies for the years 2000-2006 have been used in this evaluation. Assuming that local self-governments were able to identify their needs in their regional strategies best, we have analysed “weaknesses” and 110 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. “threats” specified there (Annex 2) and we have compared them with the objectives of Sub-Component B2. Among weaknesses and threats defined in regional strategies, problems relating to business activities in rural areas are coming to the forefront, both in the aspect of access to existing enterprises (off-farming employment) and in terms of opportunities to launch a business. The strategy of Kujawsko-Pomorskie region indicates continuing low activity and entrepreneurship with the low degree of rural population’s education and a low level of business activity development, with a weak diversity of business activities in rural areas. On the other hand, the strategy of Lubelskie voivodship indicates entrepreneurs’ little commitment to develop their businesses and population’s, especially young people’s lack of will to launch their own businesses, new entrepreneurs’ low education and lack of business incubation. The strategy of Podkarpackie voivodship stresses the absence of alternative sources of income, particularly from agri-tourism and the strategy of Zachodniopomorskie region mentions the lack of any entrepreneurship support system. The low level of rural population’s qualifications and education was the second problem put forward in almost all strategies. The strategy of Kujawsko-Pomorskie indicates that “education directions do not fit [...] in the labour market requirements and that the level of inhabitants’ qualifications is inadequate to the labour market needs”. On the other hand, the strategy of Lubelskie voivodship stresses farmers’ low education level, and in case of Podkarpackie the lack of educated human resources for off-farming activity is mentioned. The strategy of Warmińsko-Mazurskie refers to inhabitants’ education which is insufficient and inadequate to needs and to the lack of high-qualified human resources. Rural population’s low wealth which creates barriers to access to the labour market was the third problem most often identified by regional strategies from the analysed territory. The strategy of Kujawsko-Pomorskie voivodship indicates population’s progressing impoverishment, particularly in rural areas; the strategy of Lubelskie – the low average wealth of rural dwellers, irrespective of their involvement in farming, while the strategy of Świętokrzyskie voivodship refers to the low per capita income and to this income’s unfavourable structure (high share of disability pensions and old age pensions), which has a negative impact on the internal demand and the development of local entrepreneurship. The group of other problems indicated in regional strategies includes unfavourable demographic tendencies, the process of population ageing, outflow of young, active and well educated people (voivodships: Świętokrzyskie and Lubelskie), outflow of qualified specialists (Małopolskie), slow pace of transformations in rural areas ("the countryside becoming a folk museum") (Małopolskie), increasing costs of education (KujawskoPomorskie), little migration opportunities (Świętokrzyskie). One may sum it up hence, that the Program objectives identify voivodships’ real problems and needs correctly and respond to them, but do not exhaust them. One might consider including pro-mobility aspects in the Program, as well as activities facilitating shuttle migration (commuting) to work outside the place of residence and activities aimed at encouraging young and educated people to stay in rural areas and/or to move there (in order to launch an own business or support local human resources, including medical or teaching staff, etc.). UNDP 2004 Report emphasises the weight of such solutions (the chapter dedicated to the labour market in farming)49. 49 W trosce o pracę. Raport o rozwoju społecznym. Polska 2004, UNDP Warszawa 2004. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 111 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. To conclude, one may asses the adequacy of Sub-Component B1 objectives to the problems of its Target areas as satisfactory. Evaluation of Sub-Component B1 Targeting “The National Strategy for Growth of Employment and Human Resources Development for the Years 2000 – 2006” was the main document on which the policy in the field of employment and human potential development was based at the moment when RDP and Sub-Component B1 were designed50. The document referred to “The Mid-Term Strategy for Agriculture and Rural Areas Development”, to restructuring programs of selected economy sectors and to other documents adopted or being discussed by the Polish Government. Additionally, the ideas included in the Strategy complied with resolutions, recommendations and guidelines of OECD and EU in the field of employment policy. Therefore, the relevance of Sub-Component B1 should be evaluated by the means of comparing its assumptions with objectives of the Employment Strategy. The main objective of the Employment Strategy was “to achieve higher commitment of population to the process of labour”. The expected result envisaged achieving the employment index of the group of population at the age of occupational activity at the level of 62-64% by the year 2006. This goal should be achieved by the means of: (i) increasing the employment through improving the quality of human resources, (ii) developing entrepreneurship, (iii) improving companies’ and their personnel’s capability to adapt to the changing market, (iv) strengthening the policy of equal chances on the labour market. Several types of activities were designed. Apart from typically educational, the proposal included firstly improving qualifications of the unemployed, i.e. early identification of needs and intervention in case of persons threatened by unemployment, promotion of vocational training, on-the-job training, partly subsidised employment and other programs for the unemployed favouring improvement of their qualifications. Secondly, the proposal envisaged improvement of the institutional support to the labour market through working out uniformed methods of collecting data about phenomena occurring on the labour market and monitoring the effectiveness of the labour market programs. Thirdly, the Strategy addressed the need to strengthen non-public institutions providing support to the labour market, such as temporary employment agencies, personal advisory institutions, non-governmental employment agency. Fourthly, development of employment agency was put forward – improving staff qualifications, work methods and techniques and providing appropriate conditions of work to employment agents (e.g. means of communication, means of transport, operating procedures). For the fifth, the Strategy provided for developing a system of “offensive”, commonly available employment-related information and improving the quality and the availability of occupational consultancy. For the sixth, improvement of labour flexibility was proposed, e.g. work organisation flexibility (e.g. teleworking), employment flexibility (i.e. a variety of employment contract types), functional flexibility (i.e. changing personnel’s tasks, redeployment), etc. For the seventh, it was pointed out that motivation should be enhanced by the means of various mechanisms of incentives and approaches to job seeking. Thus, the National Strategy is oriented toward active forms of supporting employment, it is of a comprehensive nature and assumes participation of various ministries. Besides, social partners and expert circles have been involved in Strategy implementation. 50 “National Strategy for Employment Growth and Development of Human Resources for the years 2000-2006, Ministry of Labour and Social Policy, Warsaw, 2000. 112 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Comparing the objectives and the activities of Sub-Component B1 with those from the National Strategy one can see immediately a far-reaching convergence. The redeployment program is of a multi-sectoral nature too and it proposes tools supporting active forms of fighting the unemployment, many of them being adequate to these envisaged by the National Strategy. One should note however that Sub-Component B1 is more oriented towards the “supplyrelated” aspect of human resources than their “demand-related” aspect. This means that its proposals do not include many activities that would encourage existing employers to adjust employment opportunities in rural areas. Sub-Component B1 might propose tools increasing the flexibility of labour market organisation, e.g. create incentives for employers to make forms of employment more flexible, e.g. develop remote work opportunities in order to overcome structural barriers existing in rural areas (such as transport problems). At the same time, the tools addressing the supply-related aspect might have put more stress on tools supporting human resources’ motivation and mobility. PAD51 does not provide any simulation showing the rural market’s absorption capacity in terms of new jobs creating opportunities, while many experts claim that these opportunities are limited in rural areas. It is important therefore to improve transport facilities for persons seeking a job, to enable them to travel to the nearest townships and cities. It seems that this aspect has not been taken into consideration in Program activities. To sum it up, there is a far-reaching convergence of Sub-Component B1 with the State policy in the field of employment and human capital development in implementing an active policy of counteracting the unemployment, therefore the relevance assessment is satisfactory. Evaluation of Participation in Sub-Component B1 It is difficult to carry out any precise evaluation of participation (the proportion of participants to potential beneficiaries). Although the number and the characteristics of participants are known (based on program monitoring activities), the number of potential beneficiaries is hard to estimate due to absence of any detailed statistics. Nevertheless, below, following a general analysis of beneficiaries, we shall try to estimate the number of potential beneficiaries. We are taking into consideration both the actual beneficiaries and persons with similar social and economic features who did not participate in SubComponent B1 for various reasons (discussed in the second section of this chapter). Characteristics of Sub-Component B1 beneficiaries The criteria of participation in Sub-Component B1 have been specified in detail. The following groups of persons qualifying for support have been listed: • 51 youth living in the immediate household of individuals defined under (b), individuals who the Social Welfare or Labour Offices define as economically disadvantaged and threatened with poverty, needing assistance to upgrade skills, employment and/or in business opportunities; Project Appraisal Document (PAD), Warsaw 2000. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 113 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. • individual who are formally registered as unemployed, who have been served notice that they are going to be released from employment within 90 days, and/or whose sole sources of income is an agricultural plot with three or less counted hectares52. Besides, participation would be limited additionally by limits for concurrent use of allowances and services. The interested persons could not receive allowances increasing their income (e.g. benefits when attending training, payment when performing intervention public works) under the redeployment program, if they have already been receiving other types of benefits (e.g. unemployment benefit, social welfare assistance). The persons concerned could not participate in training and public intervention works organised under programs financed from PP/R concurrently, nor could they participate in such programs if they had participated in any similar programs financed from the Labour Fund within the last 18 months. Only persons registered formally as unemployed could participate in Intervention Public Work Programs financed under the redeployment program. According to data available from MIS B1 base53, the number of persons covered by services until the end of April 2005 amounted to 73677 in total. The number of beneficiaries who completed the program successfully amounted to 70382. The unemployed together with unemployed school graduates constituted the vast majority of beneficiaries (about 93% of the total). School graduates accounted for some 5% of all beneficiaries and their share in particular services ranged from 1% in case of temporary employment, to 7% in case of business assistance centres. Proportions of genders were very balanced in all services, with an insignificant advantage of women – the share of men was 49.7% and the share of women – 50.3%. But there were considerable differences, depending on the type of services. Women used employment-related services and business assistance centres more often than men did (61% and 60% respectively), while men were more often involved in temporary employment services (84%) and business incubators (79%). As far as the education level is concerned, persons with a vocational education background and with a post-secondary professional education were most numerous (34% and 30% respectively). There were also some persons with university degrees – this group constituted 9% of all the beneficiaries. 52 53 114 RDP Operational Manual, Warsaw 2000, p. 56. Sub-Component B1 Management Information System. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Figure 25. Beneficiaries by education levels 10% 14% primary vocational 30% secondary comprehensive post-secondary, post-s. vocational 34% university 12% Source: based on MIS-B1 base as of the end of March 2005 Most beneficiaries were provided services in Świętokrzyskie voivodship – this group amounted to 22236 people as of the end of March 2005, i.e. some 17% of the total of unemployed in this region. The smallest group of beneficiaries was from Małopolskie voivodship – 4722 people, i.e. 2.2% of the total of unemployed there. Training services were provided to the largest group of beneficiaries (22691 beneficiaries, i.e. 31% of the total), while services of business incubators covered the smallest group (107 beneficiaries, i.e. 0.2%). Detailed characteristics of beneficiaries using particular services are specified in Annex 3. Figure 26. Beneficiaries of all redeployment services by regions 25000 22236 20000 15000 10000 13145 12256 9102 6846 5000 5370 4722 0 Kujawsko-Pomorskie Lubelskie Małopolskie Podkarpackie Świętokrzyskie Warmińsko- Zachodniopomorskie Mazurskie Source: based on MIS-B1 base Potential Beneficiaries of Sub-Component B1 In order to estimate the number of potential beneficiaries of Sub-Component B1, we shall use the estimated number of persons meeting particular criteria of participation in the Program. The number of unemployed living in rural areas is the first restriction on participation that can be estimated in an objective manner. According to GUS (Main Statistical Office), such population in all voivodships covered by B3 amounted to 640 thousand people in 2003. The rate of participation (the number of beneficiaries to potential beneficiaries) estimated on this basis would be 11% in general, but the participation was above the average in Świętokrzyskie and Kujawsko-Pomorskie Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 115 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. voivodships. The persons concerned could not receive unemployment benefits during Program implementation, therefore the number of unemployed without the right to benefit can be used as another basis for approximating the number of potential beneficiaries. According to this calculation, only 15% of persons who were eligible, benefited from SubComponent B1. In our attempt to approach closer some realistic number of potential beneficiaries, we compared the number of individual farms up to 3 hectares. Having no more precise statistics at our disposal (e.g. concerning the number of youths living together with the unemployed who run a farm up to 3 hectares, who would meet the eligibility criteria), we have to base the participation evaluation on the index quoted above. According to this measure, the average participation in Sub-Component B2 was close to 18% and it went much above this average in four regions. Number of unemployed living in rural areas in thous., year 2003 (B) Beneficiaries to the unemployed in rural areas (A/B) in % Unempl. Registered in rural areas, without the right to benefit – in thous., year 2002 (C)* Beneficjenci do bezrobotnych bez zasiłków (A/C) w % Individual farms up to 3 hectares (D) Beneficiaries to individual farms up to 3 hectares (A/D) in % An attempt to estimate participation in Sub-Component B1 Number of beneficiaries (A) Table 18. Kujawsko-pomorskie 11070 91300 12% 74340 15% 15870 70% Lubelskie 3404 93900 10% 85613 4% 75794 12% Małopolskie 5484 112100 5% 93830 6% 126503 4% Podkarpackie 12904 115100 11% 29740 43% 117840 11% Świętokrzyskie 21375 69100 31% 60251 35% 52432 41% Warmińsko-mazurskie 6737 82400 8% 69489 10% 8729 77% Zachodniopomorskie 5588 76100 7% 61789 9% 13527 41% Total 72562 640000 11% 472049 15% 410695 18% Voivodship *Estimated value, with an assumption that the proportion of the unemployed without benefits in the whole group of unemployed is the same in cities and in rural areas. Source: „Rocznik Pracy”, GUS Warszawa 2003 and Roczniki Województw, GUS Warszawa 2004 One may say generally, that the probable average degree of participation was between 11 and 20 percent for the whole Sub-Component B1 (i.e. about 15% on the average). It is certain that this degree varied from region to region very much. One may conclude that the most satisfactory participation was observed in Świętokrzyskie, Kujawsko-Pomorskie, Warmińsko-Mazurskie and Zachodniopomorskie voivodships. The eligible rural population of Lubelskie, Małopolskie and Podkarpackie showed a relatively lower rate of participation. Bearing in mind the proportion of funds allocated to Sub-Component B1 (about 14% of RDP), one may regard the average participation rate estimated at some 15% of the eligible persons as satisfactory in general, stressing however that the differences between regions are disturbingly high. 116 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 2.1.3 Situation prior to and after Implementation of Sub-Component B1 Sub-Component B1 Utility Evaluation We shall evaluate the program utility, comparing expected trends with those existing in fact and the difference between the regions covered by Sub-Component B1 and the remaining regions prior to and after the program. If we consider the total of employed rural inhabitants, a rapid fall of their number could have been expected both in regions covered by Sub-Component B1 and outside them as well, before the program commencement. The drop took place in fact, but fortunately it was not as serious as one might expect. One may regard it positive that the number of employed was falling slower in voivodships participating in Sub-Component B1 than outside the program area. In the years 1996-1999 the average for the total of employed amounted to 76866 inhabitants of rural areas in B1 voivodships, while it was 72546 in the remaining regions (thus, the difference amounted to 4320 employed). In the period of the program implementation, in the years 2000-2003, these figures were: 62532 in B1 voivodships and 59640 in other regions, the difference amounting to 2892, i.e. being lower by nearly a half (Figure 23). While the number of rural dwellers working in industry was growing in the 90-ies, in the years 2000-2003 it was falling all over Poland, but the dynamics of fall were higher in the regions covered by Sub-Component B1 (4.7 against 2.8%). The rate of unemployment was growing in rural areas in the first half of the 90-ies and one might expect that it would keep growing unless some preventing actions were taken. In the years 1998-2000, the average rate of unemployment was 14.2% in rural areas of voivodships participating in Sub-Component B1, while the average of the remaining regions was 11.8% (the difference amounted to 2.37 pp.). Although the average national rate of unemployment kept growing all over Poland, the growth was slower in the area covered by Sub-Component B1 than in other regions and it even dropped insignificantly at the end of 2003. In the years 2000-2003, the average rate of unemployment was 17.2% in regions covered by Sub-Component B1 and 15% outside this area, which means that the difference dropped to 2.18 pp (see: Figure 24). The 4.5% decrease of unemployment in the group of school graduates from the region covered by SubComponent B1 was a positive effect, although some drop was also observed in the remaining voivodships – and it was even larger by 6.4%. Nevertheless, in absolute terms, the unemployment of graduates was still on a higher level in voivodships participating in Sub-Component B1. The dynamics of rate of unemployment growth in the group of persons aged 15-24 was also much lower – by 4.5 pp. – than in other regions. Besides, it should be considered positive that the unemployment of the group of unemployed aged below 24 fell in the voivodships covered by Sub-Component B1 (by 2.9%), while it remained practically unchanged in other regions54. 54 Regional Data Base of the Main Statistical Office. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 117 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Figure 27. The total of employed rural population 90000 80000 B1 voivodships 70000 Other regions B1 extrapolation 60000 Extrapolation of other regions 50000 40000 1996 1997 1998 1999 2000 2001 2002 2003 Source: Consultant’s own analyses based on Regional Data Bank of the Main Statistical Office (GUS) Figure 28. Rate of unemployment in rural areas (%) 24 22 B1 voivodships 20 Other regions 18 16 B1 extrapolation 14 Extrapolation of other regions 12 10 1998 1999 2000 2001 2002 2003 Source: Consultant’s own analyses based on regional statistical yearbooks The number of job offers submitted by employers to Powiat (District) Labour Offices decreased in the whole economy in the middle of the 90-ies. The average drop of the number of job offers reached 35% in the voivodships covered by Sub-Component B1 in the years 1998-2000, while in the remaining regions – 44%. This resulted in a growth of the number of unemployed per 1 job offer. If trends of the 90-ies had continued, the number of unemployed per 1 job offer would have grown three times, but this index has begun falling since 2002 and the fall was faster in the region covered by Sub-Component B1 (the average drop by 37% in the years 2000-2003) than outside it (by 30%). Although the index of unemployed per 1 job offer was still higher in the regions covered by SubComponent B1, the distance to other regions was reduced significantly. In the years 1998-2000 the difference in the index of unemployed per 1 job offer amounted to 187 persons (to the disadvantage of voivodships covered by Sub-Component B1), while in the years 2000-20003 this difference dropped to 136 persons per 1 job offer. 118 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Figure 29. The number of unemployed per 1 job offer 1600 1400 B1 voivodships 1200 Other regions 1000 800 . B1 extrapolation 600 Extrapolation of other regions 400 200 0 1998 1999 2000 2001 2002 2003 Source: Consultant’s own analyses based on regional statistical yearbooks The dynamics of private enterprises registered in rural areas decreased much in the years 2000-2003 when compared with the period 1996-1999. On the other hand, it is positive that in the years 2002 and 2003 the decrease of the number of enterprises registered in voivodships covered by Sub-Component B1 was not smaller than in other voivodships (Table A5 in Annex 4). In the recent years attitudes of persons occupationally passive clearly changed. More positive elements can be noted in voivodships participating in Sub-Component B1 than in the remaining regions. Although in the years 2000-2003 the growth of the number of persons willing to take up job amounted to 47.1%, it reached as much as 111.5% in the regions that did not participate in Sub-Component B1 (Table A7 in Annex 1). Moreover, the number of persons passive occupationally who explained this with the fact of taking up learning and improving qualifications increased in the regions covered by Sub-Component B1, while the percentage of such persons dropped in the remaining regions (Table A8 in Annex A). Voivodships participating in Sub-Component B1 were also in a better situation in terms of those occupationally passive people who thought that they exhausted all methods of seeking job known to them. Although the number of such persons increased by 54% in the voivodships covered by Sub-Component B1, it increased much more in the remaining regions – by as much as 141.6% (Table A9 in Annex 1). Besides, the number of occupationally passive persons who were convinced of the impossibility of finding e job due to the economic recessions dropped definitely in the voivodhips participating in SubComponent B1 (by as much as 30%). In the same period, the percentage of such persons increased by 4.8% in other regions (Table A10 in Annex 1). In general, the protracting recession can be accounted for the growth of such moods, but it seems that regions covered by Sub-Component B1 were much less susceptible to them. Additionally, the number of young unemployed people was growing less in the voivodships participating in Sub-Component B1 than in the remaining regions and the number of young actively employed people was falling less (the difference in dynamics by some 0.6 pp.) (Tables A11 and A12 in Annex 1). In general, the economic situation worsened in the years 2002-2003 and this could be seen clearly when comparing trends with the situation of the end of the 90-ies. It seems Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 119 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. however that the regions participating in Sub-Component B1 were more resistant to negative factors. Although it is hard to state at this stage that the program activities were the only reason behind the levelling the negative factors in voivodships participating in Sub-Component B1, the comparison of changes that have occurred in these regions with the remaining areas enables one to expect a positive assessment of the program utility (this thesis will be verified on the basis of analysis in the second part of evaluation, based on opinions expressed by survey respondents). Evaluation of Sub-Component B1 Effectiveness Below, outcomes assumed for the redeployment program are compared with the actual performance. According to PAD, the following was expected as a result of SubComponent B155: • increased number of new jobs and of persons running their own business as compared to pre-project time56; • job placement rate of labour force redeployment institutions increased as compared to pre-project time 57; • lower rates of hidden and open unemployment as compared to pre-project time58; • higher average incomes in participating communities59. According to the program expected outputs defined in the Program Appraisal Document, that: post program employment will be a minimum of 15 after receipt of employment services, 50% for participants in retraining, 10% of employment in temporary community employment programs, 5% of those who make initial contact for assistance in starting small business will actually start a business and 90% of those entering business incubators will continue to operate a small business at the end of one year. No performance evaluation indicators were provided for the service of local development planning, as they would be difficult to evaluate due to this service’s long-term impact60. We are comparing the assumed outcomes with the monitoring tables, where the following specific indicators have been specified: • the number of beneficiaries who completed the program successfully; • the rate of beneficiaries who found a job or who started a business – by service categories; • the number of business incubators created; 55 The report evaluating implementation of Sub-Component B1 in 2002 emphasised problems relating to effectiveness requirements (“Consultancy Services in Redeployment. Report for II quarter 2002”, IMC Consulting Limited 2002). Therefore, the assumptions for indicators were amended by Annex 3.3 to the Operational Manual adopted in January 2003. 56 Program Appraisal Document, Annex 1, Warsaw 2000. 57 Annex 2, as above. 58 Ibidem. 59 Ibidem. 60 Program Appraisal Document, Warsaw 2000. 120 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. • the number of business assistance centres that were provided support. One may state in general that the indicators set for evaluation of the program outputs depart from the planned values. The average indicators for employment-related services, training services and services of business incubators reached values below the planned level -–from 12% in case of employment-related services, to 20% in case of vocational training services.61. In case of temporary employment services and business assistance centres the performance exceeded the planned indicators – by 150% and 100% respectively. It is characteristic that outcomes produced in particular voivodships vary. Below, characteristics of activities’ outcomes are presented. Figure 30. B1 performance against planned outcomes (1 = 100%) 3 2,5 2 1,5 1 0,5 0 vocational training professional advising temporary employment business assist. centres business incubators Source: Consultant’s own analyses based on MIS-B1 database 61 Based on MIS-B1 database of the end of March 2005. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 121 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Proportion of the employed in the number of persons who received services provided under Sub-Component B1 by regions Service V Business incubators Service IV Business assist. centres Service III Temporary employment programs Service II Training sercvice Service I Employment service Table 19. Planned 15% 40% 15% 5% 80% Whole B1 area 13% 32% 38% 10% 64% Kujawsko-pomorskie 17% 56% 26% 12% 82% Lubelskie 15% 12% 32% 4% - Małopolskie 12% 30% 40% 55% 95% Podkarpackie 19% 61% 29% 5% 74% Świętokrzyskie 13% 23% 70% 12% - Warmińsko-mazurskie 100% 29% 35% 7% - 8% 66% 35% 24% 27% Employd to the number of persons who recived the service Zachodniopomorskie Source: Consultant’s own analyses based on MIS-B1 database It was expected that as a result of employment-related services, the rate of beneficiaries who took up a job would reach 15%. According to data of the end of April 2005, 2820 out of 21175 persons who received the complete employment-related services found a job (i.e. about 13%)62. Passing over the exceptional situation in Warmińsko-Mazurskie voivodship63, the highest percent of employed persons was recorded in Podkarpackie voivodship (about 19%) and the lowest – in Zachodniopomorskie (8%). Figure A1 in Annex 1 shows how the proportion of employment changed in time. The number of beneficiaries who received complete training services amounted to 22174 and 7148 of them (some 32%) were thereafter employed. This service’s expected outcome was 40%. The highest percent of people who got a job was recorded in Zachodniopomorskie voivodship (66%) and the lowest – in Lubelskie (12%). Figure A2 in Annex 1 shows how these shares changed in time. It had been expected that as a result of the temporary employment services, 15% of beneficiaries would find jobs. Out of the group of 6712 persons who received a complete service, 2525 (i.e. some 38%) found jobs. Most people were employed in Świętokrzyskie voivodship (70% of persons who were provided a complete service) and fewest – in Kujawsko-pomorskie (26%). Figure A3 in Annex 1 shows how these shares changed in time. 62 The number of beneficiaries who are employed or run a business taken from reports after the effectiveness measuring period. 63 Two persons who used this service, found jobs. 122 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. It had been expected that as a result of the services supporting business assistance centres 5% of beneficiaries who received these services and launched a business or had been running it for less than 12 months. 157 business centres were supported from the Project funds, i.e. by 108 more than it had been assumed initially. Out of 20252 of those who received a complete service some 10% found a job: most of them in Małopolskie voivodship (55 %) and fewest – in Lubelskie (4%)64. 6 business incubators were set up from the Project funds (the first one in the year 2002, the next four in 2003 and one in 2004), although it had been planned originally that 7 incubators would have been created. No incubators were created in Lubelskie, Świętokrzyskie and Warmińsko-mazurskie regions. The expected outcome of this service was a 80% share of beneficiaries existing one year after entering an incubator. Presently, the average rate is 64.5%: 82% in Kujawsko-Pomorskie, 95% in Małopolskie, 74% in Podkarpackie and 27% in Zachodniopomorskie. To assess the effectiveness of Sub-Component B1, one should also analyse the share of beneficiaries of services who found a job against the number of beneficiaries planned in contracts. The highest performance index was achieved in case of business assistance centres (around 38%) and it was higher than p0lanned contract performance by 14 pp – and in training services (Table A7 in Annex 1). To sum it up, the effectiveness varied from one service type to another, but it usually was higher than 80% of the expected performance, therefore it should be considered satisfactory. 2.2 Bottom-up Analysis Analyses of Sub-Component B1 included interviews with persons involved in its implementation at the level of Voivodship (Regional) Labour Offices and questionnaire surveys made in two selected voivodships with: • Project beneficiaries, • Potential beneficiaries and • Personnel of Voivodship Labour Offices. 14 interviews with persons involved in Sub-Component B1 implementation at the level of Voivodship Labour Offices were made (two in each of the voivodships); 100 questionnaire surveys of persons involved in Sub-Component B1 implementation at the level of District (Powiat) Labour Offices were conducted (in all voivodships covered by B1); 700 questionnaire surveys of B1 service beneficiaries took place in selected voivodships: Kujawsko-Pomorskie, representing the north-western region of Poland (more precisely, in powiat of Bydgoszcz, Inowrocław, Lipnów and Toruń) and Małopolskie, representing the south-eastern region of Poland (specifically, in powiat of Dabrowa, Nowy Sącz, Tarnów); 400 questionnaire surveys of potential beneficiaries in all voivodships participating in Sub-Component B1. 64 Such a high level observed in Małopolskie voivodship resulted from the fact that not a large part of thise service beneficiaries received a complete service (224 persons out of 616 beneficciaries from the beginning of service delivery, from final reports). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 123 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 2.2.1 Project Costs and other Outlays Identification and Measuring The budget allocated to Sub-Component B1 was the second largest under the whole RDP budget, after Component C (it accounted for some 15% of the entire RDP cost)65. According to the plan, the distribution of costs between the parties financing SubComponent B1 was as follows: • Information about economy and labour market – USD 0.51 million (World Bank USD 0.5 million, Polish Government USD 0.1 million); • Labour redeployment services – USD 43.15 million (World Bank USD 22.2 million, Polish Government USD 20.95 million); • Variable costs – USD 1.18 million (Polish Government – USD 1.18 million) As a result of funds re-allocation, the costs planned for Sub-Component B1 were as shown in the table below. Moreover, the table contains information about the real utilisation of funds as of 31 December 2004. Table 20. Budget of Sub-Component B1 (as of 31 December 2004)* Plan PLN Item Consulting services and labour market surveys training in Performance PLN Performance / Plan % 2 029.3 2 029.3 192.1 192.1 39.6 39.6 100.00% Consulting services and training for the purpose of Redeployment Program implementation 10 047.4 9 985.0 99.38% Equipment necessary to implement the Redeployment Program 612.8 612.8 10.00% Redeployment Program services 116 836.4 114 597.6 98.08% Total 129 757.6 127 456.4 98.23% Wyposażenie niezbędne prowadzenia badań rynku pracy do A car for the Ministry of Economy and Labour Project Team 100.00% 100.00% * The rate of exchange has been accepted at the level of 1 EUR = 3.95 PLN. Source: Appendix to Annex no 2 to the Agreement of 21.03.2001. According to the latest data (of 17.05.2005), the real participation of particular subjects was: 51.3% from the WB loan and 48.7% from domestic funds. According to the last Quarterly Report of Project Management available (no 17), since the beginning of the Project, the total utilisation of loan and co-financing funds from the Polish part accounted against the plan covering the whole period of Project implementation accounted for 90% in case of Sub-Component B166. According to the latest data obtained from PCU directly (17.05.2005), the degree of loan absorption reached 99.43% of the 65 66 124 Project Appraisal Document (PAD), Warsaw 2000, p.60. Quarterly Report of Project Management no 17, JKP Warsaw 2005, p.15. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. budget planned after re-allocations. Costs of carrying out the survey (economic and labour market information) and costs of labour redeployment services were lower than planned.67 To be able to assess costs, it is important to compare unit costs by services and by voivodships. Except for the employment-related services, all other services exceeded the unit cost values (per beneficiary) planned for them. Business incubators were most expensive in terms of the real unit cost against the planned one – the costs were by nearly 50% higher than planned. Table 21. Average cost of Sub-Component B1 services per beneficiary (PLN) Employment Training Temporary employment Business centres Incubators Planned cost 332 2146 5140 670 19165 Real cost 321 2398 5339 794 28125 Performance / plan (%) 97% 112% 104% 119% 147% Service Source: Consultant’s own analyses based on MIS-B1 base. The highest average cost per beneficiary was achieved in Warmińsko-Mazurskie voivodship, despite the fact that this was one of the three regions (beside Lubelskie and Świętokrzyskie), where the most expensive service, i.e. business incubator, has not been offered at all. This results from the very high cost per beneficiary in the group of employment-related services, caused by the very low number of beneficiaries participating in this service68. Figure 31. Average cost of Sub-Component B1 services in PLN per beneficiary by voivodships 3500 3000 2500 2000 1500 1000 500 0 Kujawsko-Pomorskie Lubelskie Małopolskie Podkarpackie Świętokrzyskie Warmińsko- Zachodniopomorskie Mazurskie 67 The difference between the performance and the plan results first of all from changes in USD rate of exchange. One should expect therefore that the costs of surveys exceeded the planned value much more. 68 Only 2 persons were provided this service in Warmińsko-Mazurskim voivodship, this fact causing that unit costs were 3.5 times higher than this service’s average for all voivodships. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 125 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Source: MIS-B1 base The figure below presents unit costs in a breakdown by voivodships and services as a percent of the average cost for the given services. Employment-related services were most expensive in Lubelskie (about 350% of the average for all voivodships). Besides, the temporary employment service was also most expensive in Lubelskie (about 150% of the average). Figure 32. Differences in the average unit cost (PLN thousand) in SubComponent B1 by voivodships and services 4 3,5 3 2,5 Employment Training Temporary training 2 Business assist. centres Incubators 1,5 1 0,5 0 KujawskoPomorskie Lubelskie Malopolskie Podkarpackie Świetokrzyskie WarminskoMazurskie ZachodnioPomorskie Source: based on data from Report of May 2004, IMC Consulting Limited, June 2004. Additional and alternative costs Survey respondents – persons involved in Sub-Component B1 implementation at the level of Voivodship Labour Offices indicated that there were some additional costs resulting from the need to employ additional personnel in Voivodship Labour Offices to help with the Sub-Component implementation. On the other hand, having gained experience when working on the Project, these persons were useful later on, at other EUfinanced projects and namely these persons are now assigned responsibility for implementation of new projects financed from structural funds. The majority of personnel of District (Powiat) Labour Offices (67.7%) declared that no additional costs were incurred and only 6.3% of them declared that such costs had to be paid. As far as additional costs beneficiaries might incur due to participation in the redeployment project are concerned, only 38% of them were able to express any opinion on this issue. But 55% of these persons declared that they did not incur any costs at all, 44% declared travelling costs and 1% - costs of taking exams. Answer to the question, whether the Project target group members who were provided services on a free of charge basis would pay for it is one of indicators enabling one to evaluate the effectiveness of outlays (if a vast majority of respondents declared the possibility of self-financing, a suspicion might exist that some expenses were 126 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. unnecessary, i.e. the economy was ineffective69). According to respondents’ answers, a vast majority would not participate in the Project (“definitely not” or “rather not”), if they had to pay for it (this response was given by 67.3% of the respondents). There was a group of 201% persons however, who declared that they might pay for the program. A more detailed analysis of responses shows that the most numerous groups of persons who “would definitely not participate” if they had to pay for the program consisted of persons aged 40-59 (45.5% of respondents), with a primary education (47.3%), living in one-person households (47.1%), with the income up to PLN 1000 (35-38.5%). Figure 33. Would you participate in the Program, if you had to pay for it? 90,0% 80,0% 70,0% 60,0% 50,0% 40,0% 30,0% 20,0% definitely not PLN 1001 - 1250 PLN 751 - 1000 PLN 501 - 750 PLN 251 - 500 up to PLN 250 single person’s household aix persons’ household and more university primary secondary (comprehensive) vocational) aged 40-59 aged 25-39 aged 18-24 man woman malopolskie 0,0% kujawskopomorskie 10,0% rather not Source: Consultant’s own analyses based on survey made by PBS in March 2005 As far as the level of costs the beneficiaries would be ready pay, they most often were not able to specify this (35.2% of respondents), 12,9% declared up to PLN 50, 11.3% from PLN 51 to 100, 7% PLN 101-200 and 4.8% more than PLN 200 (the rest of the respondents declaring PLN 0). The rates declared by the respondents were strongly correlated with the household’s net income per capita. To sum it up, the lowest net costs were incurred in those voivodships and in case of those services, where the largest number of beneficiaries were and this may also be regarded as the given service’s success indicator. The Project generated no additional costs on any large scale and this fact should be assessed as positive. Moreover, its economy seems also satisfactory in general. 2.2.2 Project Outputs and Outcomes Identification and Measuring Outputs generated under Sub-Component B1 Each of the services provided specific products, according to what was specified in the Instruction. 69 The idea behind these programs is to fill the gaps, instead of replacing private initiatives. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 127 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Outputs resulting from training services The training services included vocational training, general training and literacy training, formalised training in small business running. Depending of the given voivodship employers’ needs, various types of training were implemented, e.g. training for heavy machinery operators or computer training. Outputs resulting from employment-related services Professional and social advisory services, providing information about the labour market, assessment of skills, abilities and interests, job seeking programs and job clubs, job exchange and employment services, resettlement services. Outputs resulting from temporary employment services The temporary employment services included: environment reclamation/treatment, reconstruction of public infrastructure, support to social and public institutions (e.g. schools, nursing homes, health centres). Outputs resulting from business assistance centres: Accountancy, financial, legal, marketing and sales-related services, assistance in dialogue with the local administration and other consulting services provided to entrepreneurs intending to launch a business or running a business for 12 months. Outputs resulting from business incubators: Premises and facilities, technical assistance, joint use of services and equipment. Outputs resulting from local development planning: Subsidies for planning and developing proposals of studies and promotion materials, local business potential studies, preparation of promotion materials on the local business potential and investment projects. Evaluation of Sub-Component’s net outcomes and outputs External outcomes Questionnaire surveys of persons involved in Sub-Component B1 implementation at the level of Regional (Voivodship) and District (Powiat) labour Offices show that the following can be considered to be the most important external effects of the programme: • beneficiaries’ and service providers’ increased confidence in assistance programs, • strengthened position of local service providers, • improved public infrastructure, mainly due to the temporary employment service, • experience useful in applying for other funds gained (e.g. support from EU funds). All persons involved in implementation of Sub-Component B1 at the level of Voivodship Labour Offices and covered by the survey indicated the increased confidence of beneficiaries and service providers in assistance programs. The respondents noted also that as Sub-Component B1 was progressing, the number of potential beneficiaries reporting to the program was increasing too. Many beneficiaries learned about SubComponent B1 from persons who had already participated in RDP. This is confirmed by 128 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. questionnaire surveys of beneficiaries, where a significant percent of respondents had learned about Sub-Component B1 from their relatives or friends (17.5% in case of training, 27.3% in case of temporary employment, in case of employment-related services – 13.2% and in case of business assistance centres – 9.3%) and 43% of the respondents from Powiat (District) Labour Offices stated that more beneficiaries would have benefited from the program in the past, if there had been enough places and in the future – if an opportunity like this emerged. All persons involved in implementation of Sub-Component B1 indicated also that the service providers’ standing strengthened. Due to the new, decentralised approach, namely – the service projects being prepared by service providers instead of labour offices – the commitment of local service providers increased much. Service providers gained experience in project writing and management, which is indispensable now, when implementing the European Social Fund projects for example. Apart from renovations of such public utility facilities as firehouses for example, a very positive effect of the temporary employment service in the form of improved land melioration was indicated. In the opinion of one of the respondents, the positive external effect was so significant in Kujawsko-Pomorskie voivodship, that the temporary employment service was confused with an assistance program intended to improve the land melioration. The Rural Development Project envisaged a potential synergy between Sub-Component B1 and Component A. It was assumed that persons trained under Sub-Component B1 would wish to launch their own businesses and would be able to apply for micro-loans and grants. But as there was a delay in Component A activities, the effect of synergy was much weakened. For example, the start of the business assistance centres service was suspended for one year with regard to the delay in Component A implementation. Moreover, the RDP construction assumed a potential synergy between Sub-Component B1 and Sub-Component B3. The objective of Sub-Component B3 was to increase the effectiveness and the efficiency of local and regional administration units in absorption of investments. All respondents indicated that the experience gained by labour offices and self-governments when preparing proposals and when implementing projects under SubComponent B1 helped in absorption of EU funds. The local development planning service was of a greatest merit in this respect. For example, powiats ordered service providers to prepare feasibility studies of particular projects as an element of the local development planning services. These studies were then used as an element of applications for structural funds. None of the respondents indicated any negative external effects of Sub-Component B1, such as re-allocation effect or the effect of employees being pushed out by participants of the program. 2.2.3 Sub-Component B1 Cost and Benefit Analysis It is difficult to assess economic effects of surveys on the labour market and of the local development planning service, due to these tasks’ long-term nature. As far as the remaining services of Sub-Component B1 are concerned, one may make an approximate comparison of costs of creating a new job placement with the benefits resulting from employing an additional person. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 129 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. According to MIS quarterly reports, the expected cost of creating one job placement was PLN 15600, while the real cost amounted to PLN 10169 (as of 30 September 2004).70 Our computations show that excluding the local development planning service, the average cost of creating one job placement amounted to PLN 8327 for Sub-Component B1 services71 (as of the end of March 2005). Bearing in mind how diversified the services of Sub-Component B1 were, one should analyse costs per one employed for particular services. The highest cost per one employed (newly launched business) was observed in business incubators and the lowest – in employment-related services. The table below shows costs of creating one job placement in a breakdown by services. Table 22. Average cost of services under Sub-Component B1 per one employed beneficiary (PLN) Service Employment Training Temporary employment Business assistance centres Incubators Cost / employed beneficiary 2519 7614 16214 8168 35462 Source: Consultant’s own analyses based on MIS-B1 base Direct benefits resulting from implementation of Sub-Component B1 include first of all effects in the form of new job placements and newly launched businesses. These effects translate into such economic benefits as reduction of the total of unemployment benefits paid out, increase of the Budget income from income tax and turnover tax. According to computations presented in section IV.6 (Analysis of social and economic effects of the Project), the value of increasing Budget incomes as a result of employing one unemployed person in 2003 ranged from PLN 3263 to PLN 10533 per year, depending whether the individual was paid the minimum salary or the average one. On the other hand, the average annual cost of Budget support for one unemployed was estimated at PLN 1319. Thus, the total benefits for the Budget resulting from reducing the unemployment by one person ranged from PLN 4582 to PLN 11852. Consequently, this amount can be regarded as the benefit of creating one new job placement. This would also be a benefit in case of persons taking up an additional off-farm job. Additionally, in case of business assistance services and business incubators, one should include benefits resulting from employment of additional persons in a newly-launched business (or in a company supported by the incubator). The outcomes of interviews conducted with the persons involved in implementation of Sub-Component B1 show that in case of these two services, the benefits may be of a long-term nature. In many voivodships, despite the fact that the process of services ended, business assistance centres and business incubators set up under the Project continued their activity. When evaluating the benefits, one should additionally consider positive side effects resulting from activities of Sub-Component B1 (these effects have been described in section 2.2.2.). Unfortunately, it is impossible to estimate their value at this stage of analysis. The outcomes of interviews with the persons involved in implementation of SubComponent B1 show that these effects were particularly significant in case of the temporary employment services. 70 Inception Report. RDP Quantitative Evaluation, 17 January 2001. p.53. 14505 beneficiaries were employed under 5 types of services (except for the local development planning). The contractual amount for these five services amounted to around PLN 120.79 million (MIS-B1 database). 71 130 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. A comparison of costs of creating a new job placement with potential benefits shows that the costs were relatively low in case of the employment-related services72. They were also low in case of training services and business assistance services. Considering the significant positive external effects (such as improved public infrastructure), the cost of creating a new job placement under the temporary employment service is justified from the economic point of view. On the other hand, it seems that the costs incurred in case of business incubators, when compared with possible benefits, are too high. Moreover, the interviews with the persons involved in implementation of Sub-Component B1 show that this service may bring more benefits in cities instead of rural areas, where shortage of premises for running a business in not one of the most important barriers to launching or continuing a business. 2.2.4 Long-Term Impact of Sub-Component B1 Considering the short time that passed from the end of Sub-Component B1, it is not possible to evaluate the sustainability of its effects in full. Latest contracts were signed with providers of services in the year 2004. Thus, it was impossible to evaluate, how long the job placements and the companies would survive. But, the interviews and the questionnaires enable one to indicate some elements that will have a long-term impact. The change in beneficiaries’ attitude should certainly be regarded as a long-term positive effect and this fact was emphasised by all regional consultants who were interviewed. Strengthening such local service providers as local consulting companies or NGOs will also have a positive long-term impact. Studies prepared under the local development planning services represent another positive effect of this nature. They can be used to stimulate and attract investors and to absorb structural funds. 2.2.5 Activities of Sub-Component B1 Services of Sub-Component B1 were provided through local service providers selected by Regional (Voivodship) Labour Offices. Implementation of Sub-Component B1 commenced on 1 October 2001. There were delays in Sub-Component B1 implementation from its very beginning. Tenders to conduct economy and labour market surveys were beyond the budget, therefore the scope of tasks was reduced and the deadline for choosing a contractor was postponed. The contract for surveys was signed in December 2001 with Polska Sieć Doradców Biznesu company (PSDB). The proper phase of the labour market survey began in October 2002. By the end of March 2003 a pilot survey in Lubelskie voivodship and the first series of surveys in all seven voivodships were conducted under the task titled “Labour Market Survey”. By the end of September 2003 all surveys were conducted and a final report was prepared73. At the end of the year 2001 a procedure of selecting an entity responsible for technical assistance provision was taking place. In October 2001 a contract was signed with IMC Consulting Ltd. Management Information System of Sub-Component B1 (MIS B1) was started at the turn of May and June 2002. 72 If an individual who took up a job as a result of this service had been unemployed, the very benefits resulting from non-payment of the unemployment benefit for a period of 6 months (assuming the basic monthly salary of PLN 504.20) would be higher than costs of creating a job placement. 73 According to the Inception Report. RDP Quantitative Evaluation, 17 January 2001. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 131 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. At the end of August 2001 regions implementing Sub-Component B1 were issued a decision of the Ministry of Finance granting them funds from the reserve of purpose for financing projects within the frames of Sub-Component B1. In the fourth quarter of 2001, competition procedures for selecting service providers were launched in voivodships. Voivodship Labour Offices, in co-operation with regional consultants, prepared bidding documents and commenced evaluation and selection of service providers. By the end of December 2001, the total of 20 contracts for temporary employment, 45 vocational training contracts and 5 business assistance contracts were signed. By the end of 2001 Redeployment Services commenced in the first regions. Difficulties with transferring funds for tasks caused that at the beginning of 2002 the process of signing service contracts was suspended in particular voivodships. The process was resumed in the second quarter of 2002, but difficulties with timely transfer of funds continued until the third quarter of 2002 and this caused service providers’ negative perception of the Project. Additionally, the assumed effectiveness indicators turned out to be unachievable in some cases. This was why the rules of Sub-Component B1 implementation were amended from January 2003 on. These changes allowed for more efficient implementation of the Project and for a better utilisation of funds. After changes adopted in January 2003, the Project was continued without any significant problems. At the end of March 2005, 2206 contracts were signed for all services of Sub-Component B1 in voivodships and 2187 out of them were completed. The total value of all contracts signed amounted to PLN 132.7 million74. Figure 34. Schedule of Sub-Component B1 Source: Consultants own analyses Evaluation of activities Implementation of Sub-Component B1 was much delayed. The first contracts were signed no sooner than in 2001. Due to procedural problems at the beginning of SubComponent B1, significant delays in payment of funds occurred, causing service providers’ negative perception of the Project. The delayed execution of the labour market survey should also be assessed negatively75. The surveys were carried out in two 74 According to MIS-B1 base. This was also indicated by respondents involved in implementation of Sub-Component B1 at the level of Voivodship Labour Offices. 75 132 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. phases, in the years 2002-2003, i.e. concurrently with the services. Thus, they could have been used in provision of services in the years 2003 – 2004. Consequently, one of the survey purposes (namely, supporting the process of redeployment services) has not been fully achieved. 2.2.6 Case Study for Kujawsko-Pomorskie and Małopolskie voivodships Kujawsko-Pomorskie voivodship represents the north-western region and Małopolskie voivodship – the south-eastern region. The study was prepared on the basis of the questionnaire survey and interviews. The above voivodships were selected due to the variety of problems in their rural areas and the fact that all services of Sub-Component B1 were implemented there. Besides, the choice was determined by the possibility of reaching Project beneficiaries, which was impossible in Zachodniopomorskie voivodship for example76. 4 interviews were made in Kujawsko-Pomorskie and in Małopolskie with persons involved in Sub-Component B1 implementation at the level of Voivodship Labour Offices (two in each voivodship); 30 questionnaires were responded by persons involved in SubComponent B1 implementation at the level of Powiat Labour Offices (14 in KujawskoPomorskie and 16 in Małopolskie); beneficiaries of Sub-Component B1 responded to 700 questionnaires (400 and 300 respectively) and potential beneficiaries responded to 140 questionnaires (50 and 90 respectively). We consider as potential beneficiaries such individuals who had been entirely eligible to participate in the Redeployment Project, but who did not participate in it for various reasons. These persons constitute a reference group for real beneficiaries in terms of their effectiveness in job seeking, improving qualifications and occupational redeployment. The unemployed accounted for 80.6% of this group of our sample, 15.6% feared that they would lose job soon, a farm up to 3 counted hectares was the only source of income for 4.3% of the respondents and in case of 4.9% parents were unemployed or threatened with unemployment (1.6%) or a farm up to 3 counted hectares was the only source of their parents’ income (2.2%). General Evaluation of Sub-Component B1 and of Particular Services The beneficiaries gave very high marks to all service types. More than 80% expressed a positive opinion about the Project. The employment-related services were given most positive marks (as much as 87.6% of beneficiaries regarded is as good or very good), the business assistance centres took the second position (82.2%), being followed by training services (80.8%) and temporary employment (80.3%). Respondents from Powiat Labour Offices involved in Sub-Component B1 implementation were also of a positive opinion – 78.8% of officers gave it “good” or “rather good” marks. As far as particular services were concerned, most officers gave a positive mark to the training services (86.5%) and fewest – to the business incubators (14.6%), but in the latter case as much as 75% were not able to evaluate this service at all and this is where the low percentage of positive marks resulted from. On the scale from 1 to 5, where 5 was the best mark, services were given: 76 According to information of PBS, the Voivodship Labour Office in Zachodniopomorskie voivodship does not have teleaddress data of B1 beneficiaries and the responsibility to collect it had been delegated to service providers, who ignored the issue. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 133 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. vocational training 4.13, business assistance centres 4.00, local development planning 3.90, temporary employment 3.81, business incubators 3.75, employment-related services 3.72. Figure 35. Ranking of services in B1 Vocational training 4,2 4,1 Local development planning 4,0 Temporary employment 3,8 3,9 3,6 3,8 3,4 Business incubators 3,7 3,8 Employment-related services 4,0 Business assistance centres Source: Consultant’s own analyses based on PBS survey of March 2005. Thus, the general opinion on the Project was good and the average mark was close to 4 (3.96). Evaluation of Sub-Component B1 Relevance to Powiat Needs Officers’ assessment of the program relevance seems to be confirmed by the outcomes of analysis which was carried out earlier. 74% of the respondents stated that the services responded to needs of the powiat to a high degree and much less persons, 17.7% were of the opinion that Sub-Component B1 responded to these problems to a low degree or not at all. Similar evidence is provided by interviews with persons involved in SubComponent B1 implementation at the level of Regional (voivodship) Labour Offices. According to these opinions, the situation on the labour market before the program had been very bad due to the high level of unemployment, the low level of qualifications and the rural population’s low activeness. This is why they assessed the utility of a program like this highly. Evaluation of Effectiveness of Sub-Component B1 Objectives Achievement Specific objectives of Sub-Component B1 were: (i) to improve qualifications, (ii) to provide employment, (iii) to adjust the human capital to requirements of the local labour markets. As far as the first objective is concerned, the effectiveness of improving inhabitants’ qualifications by the means of the programme was relatively high in the opinion of powiat officers – 66.7% of them stated that the qualifications improved when compared with the situation in 2000. Beneficiaries’ opinions seem to confirm the same. In case of such services oriented towards retraining and improving qualifications as training and employment-related services, 73.5% of the Training services beneficiaries stated that they gained new qualifications, 18.1% improved their previous qualifications and only 3.1% did not gain any new skills or qualifications. In case of employment-related services these proportions were 67.2%; 32.8% and 0% respectively. 134 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. In case of the second objective, 77.1% officers of District (Powiat) Labour Offices stated that Sub-Component B1 helped beneficiaries to adapt to requirements of the local labour markets (8.3% out of them indicated that this was to a very high degree). They were less certain, whether this adaptation would be effective in case of requirements of more distant employers (1% were of the opinion that the program helped very much and 51% that it rather helped). The beneficiaries were slightly less optimistic in the assessment of their chances on the labour market, as 48% of them stated that their chances did not change as a result of Sub-Component B1, but nearly as many respondents (47.8%) concluded that their chances increased. Only 1.2% claimed that their chances decreased. Responses varied, depending on the service type. Temporary employment gave least chances for finding a job (72.2% of the beneficiaries stated that their chances did not change), while business assistance centres increased these chances most (61.7% of this service beneficiaries were of this opinion). Rather grew Didn’t change Rather decreased Definitely decreased Hard to say Beneficiaries’ assessment of how their chances on the labour market have grown Definitley grew Table 23. Training 7,2% 39,6% 50,7% 0,6% 0,6% 1,4% Temporary employment 2,0% 25,3% 72,2% 0,5% Employment related service 15,7% 39,7% 36,4% Business Assist. Centres 15,9% 45,8% 32,7% Average 10,2% 37,6% 48,0% 8,3% 0,5% 1,9% 3,7% 1,2% 4,5% Source: Consultant’s own analyses based on survey outcomes As far as Sub-Component B1 effectiveness in achieving the objective of finding a job on local labour markets is concerned, 10.4% of the officers stated that the program helped definitely and as much as 70.8% that it rather helped (18.8% were of the opinion that it had no effect on the chances to find a job on more distant labour markets). Most of the officers were not able to specify the percentage of beneficiaries who found a job as a result of the program in fact (45.8% stated: it is hard to say). In the group of these persons who managed to quote an estimated percentage of beneficiaries who found a job, 6.3% stated that more than 75% found a job, 10.4% stated that 50-75% and 12.5% that 40-50% found a job and 5.2% that 30-40%. The beneficiaries expressed different opinions about their job seeking experience. Around a half of them (47.3%) did not take up a job after the program, nor did they launch their own business, but the effects of some services were very good. For example, 32.3% of those who attended vocational training declared that they got a job immediately and in the nearest area, 13.4% found a job later (i.e. more than 3 months after the program completion). The lowest proportion of persons found a job in case of the temporary employment services, but 19.2% out of them found it immediately and in the nearest area, while 14.1% - later, but also in the nearest area. 62.6% did not find any job, nor did they launch their own business. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 135 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Job found later outside the nearest area Own business launched No own business launched, no job found 13,4% 1,4% 1,4% 44,8% Temporary emplyment 19,2% 1,5% 14,1% 1,5% 1,0% 62,6% Own business continued No own business launched, no job found Business launched later in the nearest area Job found later in the nearest area 1,7% Job found later in the nearest area 32,3% Business launched immediately in the nearest area Vocational training Job found immediately in the nearest area Job found immediately outside the nearest area Evaluation of employment effectiveness after the redeployment services Job found immediately in the nearest area Table 24. Employment-related services 16,5% 37,2% 1,7% 1,7% Business Assistance Centres 11,2% 22,4% 13,1% 4,7% 43,0% 10,3% 38,3% Source: Consultant’s own analyses based on survey outcomes The Project assumed also own business launching, but most of the officers were of the opinion that it did not bring any effect in the form of launching one’s own business in the area of residence (56.3%), although 34.4% stated that it definitely or rather helped launch a business in the area of one’s residence. As far as launching a business outside the area of residence (with the need to move), officers of District (Powiat) Labour Offices stated (80.2%) that the redeployment program had no influence on this and only 9.4% were of the opinion that it helped. Estimating the percentage of individuals who launched their own business turned out to be a difficult task – 71.9% of the officers stated that it is hard to say, 11.5% indicated 0%, 8.4% - 1-10% and the same proportion indicated the rate above 10%. On the other hand, the beneficiaries expressed positive opinions about two types of services that were typically business-oriented. 37% launched their own business having participated in the business assistance centres services, i.e. the proportion was nearly the same as in case of those who did not take a job or a business activity 38.3%). Additionally, as much as 24.3% found a job in the nearest area, although they did not launch their own business. In case of employment-related services, the rate of employment was lower – 48% declared that they did not take up any job or business activity, but as much as 37.2% launched an own business in the nearest area. 136 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Perecentage of Powiat Labour Offices officers’ responses in % Figure 36. Percentage of individual who found a job according to opinion of District (Powiat) Labour Offices’ personnel 14,0 12,0 10,0 8,0 6,0 4,0 2,0 0,0 do 10 % 11 - 20 % 21 - 30% 31 - 40% 41 - 50% 51 - 75% > 75% Percentage of persons who found a job Source: Consultant’s own analyses based on survey outcomes As far as the most important issue is concerned, i.e. new employment, is concerned, 47.9% of respondents from District (Powiat) Labour Offices stated that new job were created indeed (10.4% out of this group were definitely convinced of this). On the other hand, 29.2% stated that rather no new jobs were created and 4.2% that definitely no new jobs were created. 18.8% of officers indicated that the newly-created jobs had nothing in common with the redeployment program. In terms of objectives effectiveness, the question whether participation in SubComponent B1 was of any real importance for finding a job was of a key significance. The beneficiaries who really took up a job after the program expressed rather uniform opinions in this respect. The beneficiaries stated that it was easier for them to find a job as a result of all Sub-Component B1 services (this percentage ranged from 51.4% in case of temporary employment up to 69.7% in case of business assistance centres). But at the same time, there was a relatively high percentage of those who stated that the Sub-Component B1 did not contribute to the fact of finding a job (there were 48.6% of such responses in case of temporary employment and 28.8% in case of business assistance centres’ beneficiaries). Only 0.5% of those who took up a job after the program stated that the redeployment services hampered this. It is also important that – and this provides an evidence to services’ effectiveness – that in most cases those beneficiaries who managed to find employment, got a job which was adequate to the service type they participated in. In case of training, 63.1% of respondents declared that the type of training they attended was adequate to the job they found thereafter and this thesis was also confirmed by the high percentage of business assistance centres’ beneficiaries (68.1%). Moreover, 41.7% of temporary employment beneficiaries stated that they were offered a job at the place where they were temporarily employed. What is interesting, officers from District (Powiat) Labour Offices were less enthusiastic in their assessment of the effect of Sub-Component B1 on employment opportunities: 43.2% of them thought that the program had no or very little (insignificant) influence here, but at the same time 18.2% stated that many of the beneficiaries owed their new jobs to the redeployment services namely. To be able to formulate any recommendations, it is important to analyse the situation of those persons who did not find a job after the program, although they participated in it. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 137 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. The reasons were different for beneficiaries of particular services. In case of training, most persons blamed the lack of job offers in their professions (62.7%). The second most important reason quoted was too much competition from the part of persons with similar qualifications acquired as a result of training (i.e. too many persons were trained in the same field and in the same area) – this explanation was given by 13.0%. Beneficiaries of the temporary employment services referred to the fact that it was impossible to continue working at the place of their temporary employment (64.5%), the lack of job for persons with the given type of qualifications being given as the second most important reason (33.1%) and too much competition of persons experienced in the same field taking the third place (8.1%). In case of employment-related services, the same most important reason of not taking up a job was referred to – too little job offers for persons with the given type of experience (59.6%), the fact of taking up education being given as the second most important reason (19.2%), but on the other hand it is a matter of concern that 11.5% stated that they were not looking for a job at all. 56.1% of business assistance centres’ beneficiaries did not launch any own business due to the lack of sufficient funds and the next 14.6% took up education instead of a job and the same proportion of respondents declared that they had not intended to launch any business from the very beginning. The inability to find a job had similar reasons in the voivodships covered by the survey, the lack of job offers being the most important reason both in Kujawsko-Pomorskie voivodship and in Małopolskie, although more respondents indicated this as the main reasons in the first of these two regions. There were few optimists expecting to find a job or to launch a business among persons who did not find any job immediately after the program. As much as 62.5% of employment-related services’ beneficiaries and 80.6% of temporary employment beneficiaries from this group were of the opinion that their chances would not change later on (in other words, they would remain unemployed), but on the other hand not very few respondents hoped that their chances for employment would grow in time – the percentage of such persons ranged from 16.1% in case of temporary employment beneficiaries to 26% in case of employment-related services. As far as potential beneficiaries’ general assessment of Sub-Component B1 effectiveness is concerned, there were most such respondents who regarded it as average (36.4%), 21.2% considered the effectiveness to be rather good or even very good, but 18.6% assessed the program as bad or very bad even. Most negative opinions were expressed by respondents from Zachodniopomorskie voivodship and most positive – by those from Kujawsko-Pomorskie. Most aged respondents and those with a vocational education background were most critical, but there was also a considerable share of persons with university degrees in this group. In general, a significant part of the respondents knew nothing about (30.2%) or had doubts concerning the utility of redeployment programs for inhabitants of their gmina. But a half of the respondents stated that such programs were useful (including 17.5% who marked the “very useful” answer). 3.8% of respondents were of the opinion that such programs were of no use at all. Evaluation of Participation in Sub-Component B1 The surveys were expected to provide an answer to the important question, whether the program solved the problems of the given region or group of persons entirely or whether it was only a step in the right direction. Responses given by officers of District (Powiat) Labour Offices showed that 34.4% of them considered the attendance to be low when compared with the needs. At the same time however, an opinion was prevailing that 138 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. many persons (37.5% of the respondents thought so) or even very many (10.4%) benefited, considering the number of those in need. This thesis is also confirmed by opinions of persons involved in Sub-Component B1 implementation at the level of Regional (Voivodship) Labour Offices. Immediate effects (i.e. fall of the unemployment rate in the voivodship) could not be seen due to programme’s insufficient scale, considering the needs. But in general, Sub-Component B1 was mainly assessed as an assistance to people who had not been provided any support from the part of powiat labour offices before, i.e. farmers who owned small farms but were not eligible to apply for the status of an unemployed person. Identification of Sub-Component B1 External Outcomes Officers of District (Powiat) Labour Offices stated that labour redeployment services resulted in additional benefits for their powiats (38.5%), although the proportion of those who said just the opposite was the same (thus, this depended on powiat). Positive effects referred to included increased personal income of beneficiaries (56.8% of indications), increased income for gmina (45.9%), increased investment attractiveness of the area (40.5%) and development of tourism (35.1%). As much as 78.6% of the officers responding the questionnaire stated that the local development service enabled their powiat to gain experience which would be useful when applying for other EU funds, but this depended on the region – 10.7% of the officers stated that no such effect occurred and 10.7% - that there was no connection between Sub-Component B1 and the ability to apply for EU funds. In the opinion of persons involved in Sub-Component B1 implementation in both voivodships, despite the fact that immediate effects (i.e. reduced unemployment rate in the region) could not be seen due to the insufficient scale of the program in comparison with the needs, such immeasurable positive effects as beneficiaries’ increased selfconfidence and the resultant stimulation of their activeness were indicated. Additionally, such very important external effects were mentioned as: beneficiaries’ and service providers’ increased confidence in assistance programs, strengthened position of local service providers and improved public infrastructure – mainly as a result of the temporary employment services. Sub-Component B1 Efficiency Evaluation According to opinions expressed by the beneficiaries, Sub-Component B1 efficiency was definitely positive. As much as 75.7% of the respondents were of the opinion that benefits were greater than costs, 16.8% - that it was at the break-even point level and only 5.2% that the costs exceeded the benefits. Officers of District (Powiat) Labour Offices were less optimistic – only a little more than a half found the effectiveness to be positive (56.3%) and 21.2% expressed negative opinions. In a more specific breakdown into costs and benefits, most of powiat officers declared that there no additional costs were incurred with regard to Sub-Component B1, except for those planned. On the other hand, when analysing additional benefits, the proportions of those who noticed such benefits and those who did not were similar (38.5% and 36.5% respectively). As far as the first group is concerned, the following benefits were listed: 1. Increased personal income 56.8% 2. Increased income for gmina 45.9% 3. Increased investment attractiveness (e.g. inflow of new business) 40.5% Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 139 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 4. Development of tourism 40.5% Evaluation of Sub-Component B1 Outcomes Sustainability Most officers of District (Powiat) Labour Offices (52.1%) stated that effects of SubComponent B1 would be of a long-term nature (5 years), 8.3% of the respondents being definitely convinced of this. At the same time, there was a relatively large group (40.6%) of officers who thought that this would rather not be possible and additional 6.3% were definitely negative about this. Thus, the opinions were almost equally divided between sceptics and optimists. But a vast majority of the respondents stated that the effects would not last longer than one year (the opinions did not differ much from service to service and ranged from 60.9% in case of employment-related services to 71.6% in case of temporary employment, the latter being of a temporary nature anyway). Employmentrelated services were given one of the worst general marks, but on the other hand they showed the longest sustainability of the positive impact after Sub-Component B1 closing in beneficiaries’ opinion (34.8% of the beneficiaries stated that the effect would last for more than one year at least). Availability of Information about Sub-Component B1 Information about labour redeployment services should have been commonly available to anyone potentially eligible. This was the role of local authorities, among others. Most of officers of District (Powiat) Labour Offices (60.4%) assessed information availability positively, but 16.7% of them were of a critical opinion in this matter. Beneficiaries’ opinions varied much, depending on the service type. Beneficiaries of business assistance centres gave lowest marks to information availability (as much as 50.5% of them assessed it as bad, 15.9% out of them indicating the “very bad” answer). On the other hand, beneficiaries of temporary employment services were of a positive opinion in this matter (45.5% of good marks) and similarly – beneficiaries of training services (41% were satisfied with the availability of information). According to the officers, labour offices were the best sources of information (this was what 62.5% of the respondents said), with the local press taking the second place (8.3%), gmina administration the next one (6.3%) and the Internet the last one (2.1%). The beneficiaries agreed that labour offices and gmina administration offices were the best sources of information about Sub-Component B1, but they also indicated the contribution of their acquaintances and relatives in circulating information about it. For each service type a different source of information was indicated as the second most important one after labour offices. In case of temporary employment as much as 27.3% of persons learned about the program from relatives or friends, in case of training gmina administration played a significant role (21.7%), in case of employment-related services, gmina administration, friends and relatives made equal contributions to dissemination and in case of business assistance centres gmina administration was immediately after labour offices. According to our analysis, there was no information campaign in media with the broadest coverage, i.e. local television, radio and press in this case (although it is hard to say, whether press reading rate and influence would be sufficient to increase the awareness to the same degree as the two remaining media). Most of the potential beneficiaries covered by the survey did not hear about the labour redeployment program implemented under Sub-Component B1 (68.2%). The highest percentage of those who did not hear was in Małopolskie voivodship (more than 74%). As far as gender and age groups are concerned, their degree of being informed (or 140 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. underinformed) about the program was the same. But in terms of education level, the percentage of those with a low educational background was higher (74.5%) than of those with the secondary and university education among those who did note hear about the program. Similarly, the highest proportion of people with the lowest income did not know about the program (71.6%). In the group of potential beneficiaries who heard about labour redeployment under SubComponent B1, most of them learned about it from the national television – 51.7%; thereafter – in labour offices: 27.1% and 19.5% from relatives. Much less persons learned about the program in gmina administration offices, from the press and the Internet. Identification of Reasons why a Person did not Participate in Sub-Component B1 The most usual reasons why potential beneficiaries did not participate in the program were as follows: (i) insufficient information (29.7 of the respondents), (ii) disbelief hat participation in the program would change anything in the respondent’s situation (25.4%), (iii) program inadequacy to the needs (20.3%), (iv) time and cost of travelling (11%), (v) lack of vacant places was on the fifth position (6.8%). The distribution of these reasons was significantly diversified in terms of socio-economic and geographic features of potential beneficiaries. Persons living in Podkarpackie and Warmińsko-Mazurskie voivodships complained most about the lack of information; the same problem was indicated by persons aged 25-39 and persons with the primary and vocational education, living in large families with lowest income per capita. The disbelief that participation in the labour redeployment program would change anything was characteristic for persons from Kujawsko-Pomorskie voivodship, more often men than women, elderly people, with the basic vocational education, living single, with the average income per person in the family. The inadequacy of the program to the needs was mainly indicated by inhabitants of Kujawsko-Pomorskie voivodship, elderly persons, mainly with university degrees, but also – what is interesting – with the basic vocational education, also persons living single and with a low income. The time and the cost of travelling was the most important barrier for inhabitants of Podkarpackie and Świętokrzyskie voivodship. Moreover, these were women more often, persons aged 25-39, with the basic vocational education, from large families with low or average income. The lack of vacant places gave hardest time to inhabitants of Lubelskie voivodship, young people with the secondary and university education, with a relatively high income. The potential beneficiaries declared, which activities they would like to participate in under the labour redeployment services. The first positions were taken by temporary employment (as much as 80.1% declared their wish to participate) and by training (77.6%). The employment-related services were much less popular (46.9% of respondents), just as business assistance centres (34.5%) and business incubators (28.8%). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 141 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Job Seeking Methods Used by those who did not Participate in Sub-Component B1 A vast majority of respondents (mostly the unemployed) have been looking for a job for the last 4 years (i.e. in the period when Sub-Component B1 was implemented in their region), but 26.4% of them did not do this intensively (from 1 to 5 times in this period), 17.3% more intensively (from 6 to 10 times) and 16.7% very intensively (more than 11 times). When analysing data for all the voivodships, one can see that only 7.5% of the respondents did not look for a job at all. But a declaration like this was made by as much as 15.7% of respondents in Podkarpackie voivodship and 10.8% in Małopolskie (while only 2.5% in Warmińsko-Mazurskie and only 3.1% in Małopolskie). Women were those who did not look for a job much more frequently than men and – surprisingly – persons aged 25–39. The reluctance to look for a job was correlated with the education level – persons with the primary education background constituted the largest percentage of those who did not look for a job. Members of households with the highest income per capita looked for a job less frequently. Most of the potential beneficiaries did not look for help of institutions specialised in employment agency (58.2% declared an approach like this on the average), but the rest (41.8%) had some contacts with such institutions. When analysing data for all voivodships, one can see that least interest in institutional help was declared in Małopolskie voivodship (36.9% of the respondents only) and most – in WarmińskoMazurskie voivodship (55%). Gender was insignificant in seeking help of employment agency institutions, but age was of much importance. Mainly the youngest people were looking for institutional help (aged 18-24), while the oldest (aged above 65) declared least interest in this form of assistance. Education was similarly important – most of people seeking institutional help in employment had university degrees or at least a secondary educational background. Most persons interested in this for of help declared the average level of income per person in their households. Labour Offices or Employment Agency Offices were most often indicated as institutions where potential beneficiaries were seeking advice and assistance in finding a job (96.8%) and only 8.4% went to professional advisory companies. The latter were most often visited by young people with university degrees. Other forms of looking for a job used by potential beneficiaries included: using family contacts (as much as 78.4%); looking for job advertisements in local newspapers (63.9%); looking for job advertisements in national press (25.6%), in the Internet (10%) and an job fairs (4% only). Comparison of Sub-Component B1 with Other Similar Programs The labour redeployment services of Sub-Component B1 were not the only labour market-oriented program in the regions were RDP was implemented. 68.8% of officers of District (Powiat) Labour Offices confirmed the existence of similar programs, and only 14.6% stated that no such programs were implemented (the rest of respondents did not know the answer). The group of most popular programs included: PHARE 2000, 2001, 2002 (this was confirmed by 80.3 of the surveyed officers), structural programs (ESF) (confirmed by 34.8% of the respondents), SPO RZL (6.1% of the respondents), green jobs (4.5%), Alternatywa (6.1%) and other projects (known to 40.9% of the respondents). But one can clearly see that due to well formulated criteria of beneficiaries selection for RDP it was possible to reach such persons who had not participated in any other programs for the last 4 years. 79.9% of the training service beneficiaries had not participated in any similar program; in case of temporary employment this rate was as 142 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. high as 94.9% of persons who participated in Sub-Component B1; 92.6% of employment related services’ beneficiaries had not participated in any such program before; in case of business assistance centres, 91.6% of persons declared that they had not participated in similar programs. Comparing RDP with other programs, 50% of District (Powiat) Labour Offices stated that labour redeployment services of Sub-Component B1 improved qualifications to the same degree as other similar projects, while 21.2% indicated that this degree was higher and 18.2% - that it was lower. Similarly as in case of improving qualifications, the effectiveness of the project in finding a job was considered to be the same as the effectiveness of other programs (this opinion was expressed by 50% of Powiat Labour Offices’ officers); 21.2% stated that the effectiveness of redeployment services was higher and 15.1% - that it was lower. As far as the next objective achievement is concerned, i.e. adaptation to the labour market requirements, the effectiveness of RDP was also comparable with that of similar programs (this is what 48.5% of the officers stated). In case of those beneficiaries who had participated in other programs (training projects mostly), 45.2% of them declared that RDP improved their qualifications to the same degree as other programs, 24.7% - that it was better, but 20.5% thought that it was worse. From the group of persons who participated in various forms of institutional assistance (outside Sub-Component B1), most respondents were provided job counselling (34.2%), the second position being taken by training services (30.3%). 18.1% declared that institutional help was limited to access to job offers; 15.5% participated in public works; only 5.8% participated in counselling and training programs dedicated to business launching and 1.3% were provided assistance intended to help those who had already been running their own business. The proportion of those who assessed institutional help negatively was highest (41.9%), 18.7% considering it to be very poor even. On the other hand, 36.6% found that the assistance represented an average level. But every fifth person assessed the institutional help they had been provided as good or very good. In comparison with data recorded in all voivodhips, the highest percentage of persons most satisfied was in Zachodniopomorskie voivodship (16.7%) and the lowest – in Warmińsko-mazurskie (27.3%) and Lubelskie (24.0%). Usually, women were more dissatisfied than men and persons with the primary education indicated a higher degree of dissatisfaction too. The effectiveness of programs implemented concurrently with B1 seems to be rather low. 71.6% declared that they did not manage to find a job thereafter, nor did they launch any own business. Only 5.8% of the respondents stated that they found a job immediately (within a period up to 3 months) after the program and 15.5% that this happened later. Only 0.6% commenced their own business activity having been provided assistance. In comparison with data recorded in all voivodships, the best results were noted in Zachodniopomorskie, where 16.7% of the respondents managed to find a job immediately and less than a half could not find it at all. Women, persons aged 25-39 and persons with university degrees managed to find a job soonest than any other persons. Considering the effectiveness of programs similar to Sub-Component B1 redeployment services in terms of adaptation to requirements of employers in the regions covered by Sub-Component B1, as much as 81.9% stated that they did not derive any benefits. Only Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 143 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 15.5% declared, that these services helped them to adjust to the local employers’ requirements. The outcomes quoted above show that Sub-Component B1 was relatively more effective in comparison with similar programs implemented in these voivodships. 2.2.7 Conclusions and Recommendations for the future The above analysis enables one to present the most important conclusions that follow the experience gained as a result of Sub-Component B1 implementation: 144 • The analysis of objective secondary sources proved that before the year 2000 (i.e. before implementation of Sub-Component B1), the situation of regions to be covered by this project had been worse on the average than in other voivodships. What is more, the labour market-related problems had been of a long-term nature and market mechanisms had not been leading to any solution. Thus, the intervention in the form of a program like Sub-Component B1, oriented towards labour redeployment, has been assessed as justified. • The comparison of Sub-Component B1 with problems identified in regional strategies showed a far-reaching convergence indicating that Sub-Component B1 identified these voivodships’ real problems and responded to them correctly. This opinion was also confirmed by a significant part of the respondents, although there was a group of them – aged people with university degrees and low income – who indicated program inadequacy. • The analysis of situation in the voivodships covered by Sub-Component B1 in comparison with the remaining regions after project implementation showed that the situation of regions covered by Sub-Component B1 relatively improved in some aspects. One should stress however that the respondents’ general opinion seems to indicate that the scale of intervention was too small when compared with the needs, to be able to speak of Sub-Component B1 ability to solve the problems, but this was a step in the right direction. • As far as the scale of participation is concerned, it has been estimated that about 15% of persons eligible for the project participated in it on the average. The respondents, beside reporting the fact that there were not enough places to accommodate their participation, indicated also other problems that set limits to their participation, such as insufficient information, disbelief that the program could change their situation on the labour market, program inadequacy to their needs and costs of travelling. • The effectiveness of Sub-Component B1 planned outputs achievement has been evaluated on the basis of objective indicators and based on respondents’ opinions as well. Although the effectiveness varied from service to service, it exceeded 80% of the assumed performance on the average. Good effectiveness of the project is also confirmed by outcomes of interviews and questionnaire surveys. • Although the immediate general outcome of the project (i.e. decrease of unemployment in the selected voivodships) was not evident (due to the scale of SubComponent B1), respondents’ opinions indicated an important additional impact of Sub-Component B1 in the form of increased self-confidence and stimulated Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. activeness of those persons experiencing problems with employment who had not been participating in any similar programs in their majority. • It is hard to identify the sustainability, as not much enough time has passed since the project closing. The respondents’ opinions concerning the sustainability of effects varied. The beneficiaries themselves believed in it more than the officers who were responsible for the project and who had a more comprehensive knowledge of the labour market. • As far as the cost/benefit proportion is concerned, it should be stressed that some of the costs might have been distributed among the beneficiaries who declared in their questionnaires that they could pay a part of the costs of services they were provided (this would be possible with the adequately defined criteria of participation). On the other hand, one should emphasise those unintended – mostly positive. The very good construction and planning of RDP should be considered as its greatest strength – particularly, the idea to adopt the multi-sectoral approach to the complicated matter of the labour market problems in rural areas (no approach like this had existed in previous programs of this type). This approach enforced co-operation between various ministries and allowed for synergy between Components. The following should be considered to be the most important weaknesses of SubComponent B1: • Lack of co-ordination with Component A (micro-loans) which started with a serious delay and this fact affected business assistance centres and business incubators activity negatively. The RDP construction allowed for potential synergy between Component A and Sub-Component B1. It had been assumed that persons trained under Sub-Component B1 would have wished to launch their own businesses, using micro-loans and grants. As a result of the circumstances referred to, the effects of synergy were much weaker than those potentially possible. • Serious delays in disbursement of funds for service providers resulted in a reduced participation of smaller companies having no sufficient financial potential of their own. • Although Sub-Component B1 orientation towards disadvantaged inhabitants of rural areas achieved by the means of skilful formulation of the eligibility criteria was one of the greatest advantages of the project (when compared with other programs, such as PHARE), the reality of project implementation did not fully follow the guidelines. This means that the disadvantaged persons, owners of very small farms and school graduates from rural areas constituted a small percentage of Sub-Component B1 beneficiaries. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 145 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Annex 1 Supplementary Tables and Figures Table A 1 Specification of the unemployed by voivodships in the years 19981999 Łódzkie Małopolskie Mazowieckie Opolskie Podkarpackie Podlaskie Pomorskie Śląskie Świętokrzyskie Warmińsko-mazurskie Wielkopolskie Zachodniopomorskie B1 voivodships Other voivodships Differencee in (B1 – other voivoidships) 1998 B1 voivodships 1999 1999 Other voivodships 1999 Differencee in (B1 – other voivoidships) 1999 B1 change % 1999/1998 Other change % 1999/1998 Difference in change % 1999/1998 (basis points) The unemployed per job offer Lubuskie Jobs offers in thous. Lubelskie Unemployment aged up to 25 % in total Kujawsko-pomorskie 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1999 1998 1998 Rate of unemployment Dolnośląskie Graduates Region Residents of rural areas Registerd unemployment in thous.. 57,8 73,6 55,9 60,7 65 82 22 31,1 50,1 64,6 67,6 91,7 95 117,6 20,9 26,6 89,8 106,9 23,6 27,3 45,7 57,8 28,4 39,7 48 59,6 59,1 70,6 61 80,1 45,8 57,7 60,2 39,5 8,4 11,3 6,1 8,2 8,5 10,9 2,6 3,9 8 10,2 10,5 15,4 10,9 14,3 2,9 4 9,9 11,9 3,6 4,2 4,6 7,4 11,9 17,2 4,6 6,1 4,8 5,7 9,3 13,2 4,1 5,9 6,5 6,0 12,8 16 13,9 16,9 10,3 12,9 13,2 17,5 11,4 14,3 7,6 10,2 7,6 9,5 10,5 13,2 12,3 14,5 10,8 12,5 11 13,8 7,3 10,4 12,1 15,1 19,7 22,4 8 10,5 13,9 18,1 12,4 10,1 28,2 28,6 30,5 30,8 35,3 35,9 29,4 29,4 29,2 28,9 38 37,9 29,1 28,7 29,1 29,1 34,8 33,6 29,4 30,1 27,3 28,5 32,3 32,2 31,6 31 27,3 27,1 35,5 35,9 26,1 26,6 31,7 29,9 0,6 0,7 0,8 0,4 0,2 0,3 0,2 0,01 0,7 0,5 0,5 0,2 1,3 1 0,1 0,1 0,2 0,2 0,2 0,1 0,5 0,7 0,5 0,6 0,1 0,1 0,3 0,3 0,9 0,6 0,3 0,3 0,3 0,4 247 279 156 390 571 559 285 1699 206 387 250 638 153 249 709 508 916 729 392 970 175 191 298 370 808 1580 349 426 142 260 357 488 413 255 20,7 0,5 2,3 1,8 -0,1 158 73,7 51,0 8,6 8,3 15,3 12,8 31,6 30,1 0,2 0,3 615 421 22,8 0,3 2,5 1,5 0,0 194 18% 22% 24% 27% 19% 22% -0,3% 0,7% -19% -56% 33% 39% -4% -3% -2% -1,0% 38% -7% * Geometric mean ** As of 31 XII Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS (Regional Bank Data of the Main Statistical Office). 146 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 2 Balance of internal and international migration in the years 1995 and 2000 (permanent, per 1000 people) Region 1995 2000 Dolnośląskie -0,6 -0,7 Kujawsko-pomorskie -0,8 -0,5 Lubelskie -1,1 -1,4 Lubuskie -0,1 -0,6 Łódzkie -0,4 -0,5 Małopolskie 0,2 0,6 Mazowieckie 0,8 1,8 Opolskie -2,8 -3,4 Podkarpackie -0,7 -0,9 Podlaskie -1,1 -1,3 Pomorskie -0,2 0,2 Śląskie -0,9 -2,5 Świętokrzyskie -1,6 -1,6 Warmińsko-mazurskie -1,5 -1,7 Wielkopolskie 0,2 0,4 Zachodniopomorskie -0,7 -0,4 B1 voivodships -6,2 -5,9 Other voivodships -5,1 -6,6 Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 147 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 3 Employment rate in the II quarter in the years 1995-1999 1995 1996 1997 1998 1999 1995-1999 mean* 1999/1995 % change POLAND 60,8 60,4 60,8 61,1 59,2 60,5 -3% Dolnośląskie 56,3 56,4 55,7 57,5 56,8 60,4 -2% Kujawsko-pomorskie 56,9 58,2 58,7 59,2 56,9 56,6 1% Lubelskie 70,2 67,1 65,6 67,6 61,9 58,2 -2% Lubuskie 53,6 53,1 54,9 59,9 52,4 65,5 -8% Łódzkie 62,8 63,4 63,8 62,3 61,4 55,0 -1% Małopolskie 65,2 64,4 63,5 64,8 63,1 62,7 -3% Mazowieckie 64,0 64,0 65,1 65,6 63,5 64,0 -2% Opolskie 59,3 59,2 60,8 60,8 55,7 64,6 -1% Podkarpackie 66,5 65,2 65,6 64,3 60,1 59,1 -6% Podlaskie 66,2 70,4 69,3 63,1 61,2 63,8 -8% Pomorskie 57,8 55,5 56,3 57,4 57,9 65,9 -13% Śląskie 57,2 56,2 57,2 56,3 55,4 56,8 4% Świętokrzyskie 67,0 68,6 66,3 63,6 61,7 56,3 -2% Warmińsko-mazurskie 52,4 50,2 53,2 55,7 52,4 65,0 -10% Wielkopolskie 58,3 58,4 59,2 60,8 60,5 52,9 4% Zachodniopomorskie 56,8 55,2 55,9 56,9 54,8 59,7 3% B1 voivodships 61,8 60,9 61,1 61,6 58,6 60,8 -5,2% Other voivodships 60,6 61,3 61,7 60,8 58,7 60,6 -3,0% Regions Difference in basis points (B1 – other voivodships) -2,2% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS 148 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Characteristics of rural areas by voivodships in the year 2000 – comparison of B1 voivodships with other voivodships Poland Share of farms 1 – 3 hectares (overall %) Precentage of farms running off0farm business – overall % Individual farms generating income from off-farming activity Farms where the managing person has no agricultural education background Percentage of women as users of farms % Individual farms 1 – 3 hectares (in units) % Proportion of unemployed population aged mote than 15 Share of rural population % Population not related to a farm Share of employed in agriculture Poland = 100 Population related to a farm Share of agriculture in gross addev value Rate of unemployment * Gross National Product per capita Table A 4 % % % szt % % % % % Rural areas Total % 100 3.8 28 38.3 18.5 10.9 26.8 24 697330 37.8 100 8.3 43.1 26.1 Dolnośląskie 103.4 2.8 15.5 28.6 23.6 17.3 30.5 33 28739 39.1 5.8 12.7 34.9 28.4 Kujawsko-pomorskie 89.7 3.9 25.9 37.9 18.4 9.9 24.5 27.9 15870 20.8 3.2 7.2 28.9 19.5 Lubelskie 68.5 6 51.7 53.1 14.2 8.1 21.6 18 75794 33.1 8.2 5.1 40.6 24.7 Lubuskie 89.7 4 16.5 35.4 26 18.2 29.9 35 11576 42 2.9 17.6 41.3 26.8 Łódzkie 88.7 4 32.2 35.2 20.6 12.1 27.8 21.3 44449 28.1 10.3 9.6 39.7 23.5 Małopolskie 89.3 3 35.7 49.8 14.1 11.3 20.8 22.4 126503 62.2 12 10.1 56.7 32.8 Mazowieckie 151.6 3.2 24.7 35.8 15.5 8 25.7 24.4 68663 25.9 17.4 8.8 40.8 21.2 Opolskie 85.5 5.7 28 47.6 17.2 5.8 22.3 27.7 11064 32.4 2.2 9.5 28.8 22.1 Podkarpackie 71.1 3.8 46.9 59.1 18.8 18 28.3 21.8 117840 61.4 7.1 5.4 55.3 38.5 Podlaskie 74.3 7.3 45.9 41.4 16.3 3.1 30.8 21.4 14602 14.9 3.9 6.7 34.3 17.1 Pomorskie 100.6 2.6 14.5 31.9 19.3 10.1 32.3 30 10532 23.6 3.6 13.7 35.3 20.2 Śląskie 110.1 1.7 12 20.6 20.3 19 22.7 29.8 58330 60.8 6.3 11.4 58.9 33.1 Świętokrzyskie 78.2 5.8 48.6 54 16 8.6 18.7 11.7 52432 41.5 4.5 5.9 55.7 30.1 Warmińsko-mazurskie 74.5 6.8 25.3 39.9 23.7 10.9 41.9 28 8729 19.9 2.9 8.7 36.2 18.5 106.6 5.7 25.5 42.5 17.1 8.7 21.3 29.5 38680 28.5 7.2 7.7 25.8 21 98.7 4.3 13.8 30.5 22.8 22.2 33.5 30.3 13527 36.5 2.5 11.1 32.8 19.7 Average for B1 81.4 4.8 35.4 46.3 18.3 12.7 27.0 22.9 58671 39.3 5.8 7.6 43.7 26.3 Average for the Rest 101.2 4.1 23.9 35.4 19.5 11.4 27.0 28.0 31848 32.8 6.6 10.9 37.8 23.7 Wielkopolskie Zachodniopomorskie Average for: * in 2001 Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS, Charakterystyki gospodarstw rolnych w 2000 r., GUS Warszawa 2001. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 149 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 5 Entities registered in REGON system – private sector in rural areas in the years 2002-2003 2002 2003 2003/2002 % change Dolnośląskie 49321 52090 6% Kujawsko-pomorskie 42754 44906 5% Lubelskie 46779 49849 7% Lubuskie 21192 21857 3% Łódzkie 47852 51116 7% Małopolskie 86797 90819 5% Mazowieckie 102870 107678 5% Opolskie 23647 24630 4% Podkarpackie 54138 56260 4% Podlaskie 21448 22824 6% Pomorskie 40599 43461 7% Śląskie 66874 68641 3% Świętokrzyskie 31917 33761 6% Warmińsko-mazurskie 23693 25109 6% Wielkopolskie 85967 90276 5% Zachodniopomorskie 34489 36271 5% B1 voivodships 42369 44612 5,3% Other voivodships 38559 40601 5,3% Region Difference in basis points (B1 – other voivodships) 0,0% Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS 150 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 6 Balance of internal and international migration in the years 20002003 (permanent, per 1000 people) 2000 2001 2002 2003 2000-2003 Total Dolnośląskie -573 177 -684 -1219 -2582 Kujawsko-pomorskie -407 -868 -1021 -1057 -3597 Lubelskie -2969 -3643 -3840 -4576 -18062 Lubuskie -440 -812 -795 -435 -2741 Łódzkie -1107 -1339 -1407 -915 -5982 Małopolskie 2376 2644 2986 3298 13248 Mazowieckie 8825 9866 12166 13328 50912 -88 -247 -682 -540 -1579 Podkarpackie -1730 -1791 -2431 -2192 -9672 Podlaskie -1255 -1100 -1481 -1589 -6307 Pomorskie 1651 2074 1982 2000 9827 Śląskie -1652 -2353 -1639 -2909 -8947 Świętokrzyskie -2061 -2304 -2372 -2467 -11499 Warmińsko-mazurskie -2002 -2176 -2136 -1580 -9889 Wielkopolskie 1595 2430 2306 2239 10362 Zachodniopomorskie -163 -558 -952 -1386 -3492 B1 voivodships -6956 -8696 -9766 -9960 -35378 Other voivodships 6956 8696 9766 9960 35378 Region Opolskie Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 151 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 7 Persons occupationally passive due to reluctance to take up a job in the years 1999-2003 (in thousand) 1999 2000 2001 2002 2003 2000-2003 mean* 2003/2000 % change Dolnośląskie 35 27 22 19 27 23 0% Kujawsko-pomorskie 25 13 17 27 30 21 131% Lubelskie 10 7 10 21 23 14 229% Lubuskie 6 5 10 15 10 9 100% Łódzkie 31 18 18 25 20 20 11% Małopolskie 19 11 21 15 9 13 -18% Mazowieckie 45 37 53 45 46 45 24% Opolskie 8 4 21 27 19 14 375% Podkarpackie 34 23 17 12 10 15 -57% Podlaskie 2 4 4 4 19 6 375% Pomorskie 9 14 21 23 33 22 136% Śląskie 112 133 71 88 73 88 -45% Świętokrzyskie 23 23 46 57 40 39 74% Warmińsko-mazurskie 7 7 17 22 20 15 186% Wielkopolskie 31 24 33 33 50 34 108% Zachodniopomorskie 24 18 13 13 20 16 11% B1 voivodships 18 13 18 21 19 18 47,1% Other voivodships 12 12 16 17 26 17 111,5% Region Difference in basis points (B1 – other voivodships) -64,4% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS 152 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 8 Persons occupationally passive due to school/training attendance in the years 1999-2003 (in thousand) 1999 2000 2001 2002 2003 2000-2003 mean* 2003/2000 % change Dolnośląskie 275 267 250 255 256 257 -4% Kujawsko-pomorskie 207 210 201 205 219 209 4% Lubelskie 181 183 199 215 227 205 24% Lubuskie 84 82 94 110 109 98 33% Łódzkie 221 236 253 242 223 238 -6% Małopolskie 261 290 312 320 297 305 2% Mazowieckie 436 458 440 422 444 441 -3% Opolskie 94 85 82 83 91 85 7% Podkarpackie 177 177 173 199 215 190 21% Podlaskie 110 122 120 106 105 113 -14% Pomorskie 165 177 189 180 189 184 7% Śląskie 346 382 390 443 435 412 14% Świętokrzyskie 127 131 132 130 136 132 4% Warmińsko-mazurskie 130 126 114 129 139 127 10% Wielkopolskie 265 282 275 296 281 283 0% Zachodniopomorskie 131 158 175 172 161 166 2% B1 voivodships 168 176 178 187 192 183 9,4% Other voivodships 173 181 183 181 181 181 -0,3% Region Difference in basis points (B1 – other voivodships) 9,7% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 153 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 9 Persons occupationally passive due to exhaustion of all known methods of looking for a job in the years 1999-2003 (in thousand) 1999 2000 2001 2002 2003 2000-2003 mean* 2003/2000 % change Dolnośląskie 18 6 7 11 8 8 33% Kujawsko-pomorskie 6 6 14 23 19 14 217% Lubelskie 6 3 6 9 5 5 67% Lubuskie 3 2 5 7 9 5 350% Łódzkie 11 3 6 12 14 7 367% Małopolskie 4 6 13 16 9 10 50% Mazowieckie 11 10 14 28 11 14 10% Opolskie 4 3 11 18 16 10 433% Podkarpackie 3 4 8 6 6 6 50% Podlaskie 1 2 7 7 5 5 150% Pomorskie 7 5 5 7 14 7 180% Śląskie 8 13 23 18 23 19 77% Świętokrzyskie 3 7 7 7 6 7 -14% Warmińsko-mazurskie 4 5 6 7 7 6 40% Wielkopolskie 5 7 9 16 9 10 29% Zachodniopomorskie 2 9 15 12 13 12 44% B1 voivodships 4 5 9 10 8 8 54,0% Other voivodships 4 4 8 11 10 8 141,6% Region Difference in basis points (B1 – other voivodships) -87,5% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS 154 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 10 Persons occupationally passive due to conviction of impossibility to find a job in the years (in thousand) Region 2000-2003 2003/2000 mean* % change 1999 2000 2001 2002 2003 Dolnośląskie 14 9 14 17 22 15 144% Kujawsko-pomorskie 15 13 10 13 11 12 -15% Lubelskie 19 14 7 10 8 9 -43% Lubuskie 5 6 10 8 6 7 0% Łódzkie 17 29 17 16 15 19 -48% Małopolskie 11 9 22 14 12 14 33% Mazowieckie 14 24 17 8 10 13 -58% Opolskie 4 5 10 6 6 7 20% Podkarpackie 13 7 8 4 4 5 -43% Podlaskie 4 3 5 5 4 4 33% Pomorskie 12 14 13 13 10 12 -29% Śląskie 26 24 27 26 26 26 8% Świętokrzyskie 10 14 4 12 7 8 -50% Warmińsko-mazurskie 7 13 12 14 5 10 -62% Wielkopolskie 11 9 13 14 13 12 44% Zachodniopomorskie 21 12 17 20 14 15 17% B1 voivodships 13 11 10 11 8 10 -30,0% Other voivodships 8 8 11 10 9 9 4,8% Difference in basis points (B1 – other voivodships) -34,8% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 155 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 11 Persons occupationally active aged 15-29 in the years 1999-2003 (in thousand) 1999 2000 2001 2002 2003 2000-2003 mean* 2003/2000 % change Dolnośląskie 425 361 304 305 309 319 -14% Kujawsko-pomorskie 284 291 279 278 264 278 -9% Lubelskie 273 269 292 301 276 284 3% Lubuskie 147 112 142 131 133 129 19% Łódzkie 344 362 333 356 398 362 10% Małopolskie 440 426 394 439 457 428 7% Mazowieckie 550 595 612 594 629 607 6% Opolskie 129 124 108 95 100 106 -19% Podkarpackie 272 221 209 228 235 223 6% Podlaskie 160 137 139 134 120 132 -12% Pomorskie 234 249 283 242 204 243 -18% Śląskie 435 494 596 540 556 545 13% Świętokrzyskie 166 183 161 148 136 156 -26% Warmińsko-mazurskie 164 174 165 160 155 163 -11% Wielkopolskie 476 506 484 461 467 479 -8% Zachodniopomorskie 194 186 206 205 188 196 1% B1 voivodships 242 238 232 236 227 233 -4,8% Other voivodships 255 243 247 234 230 238 -5,4% Region Difference in basis points (B1 – other voivodships) 0,6% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS 156 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 12 The unemployed aged 15-29 in the years 1999-2003 (in thousand) Region 2000-2003 2003/2000 mean* % change 1999 2000 2001 2002 2003 Dolnośląskie 115 116 109 125 106 114 -9% Kujawsko-pomorskie 77 79 75 89 97 85 23% Lubelskie 74 72 82 82 72 77 0% Lubuskie 42 37 54 53 47 47 27% Łódzkie 85 101 122 111 121 113 20% Małopolskie 105 89 101 130 145 114 63% Mazowieckie 110 120 148 174 158 149 32% Opolskie 28 30 36 30 26 30 -13% Podkarpackie 90 73 74 92 78 79 7% Podlaskie 30 31 40 40 35 36 13% Pomorskie 56 75 84 86 64 77 -15% Śląskie 106 113 186 161 185 158 64% Świętokrzyskie 48 49 53 50 43 49 -12% Warmińsko-mazurskie 54 59 65 63 52 60 -12% Wielkopolskie 93 97 128 132 119 118 23% Zachodniopomorskie 56 51 69 78 68 66 33% B1 voivodships 69 66 73 80 74 73 12,0% Other voivodships 57 60 76 75 68 70 13,2% Difference in basis points (B1 – other voivodships) -1,3% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 157 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 13 Persons occupationally passive aged 15-29 in the years 1999-2003 (in thousand) 1999 2000 2001 2002 2003 2000-2003 mean* 2003/2000 % change Dolnośląskie 359 325 308 303 303 310 -7% Kujawsko-pomorskie 253 267 254 258 273 263 2% Lubelskie 220 224 238 257 264 245 18% Lubuskie 115 113 120 131 132 124 17% Łódzkie 270 283 306 299 288 294 2% Małopolskie 334 349 371 377 345 360 -1% Mazowieckie 512 536 520 505 538 525 0% Opolskie 118 105 100 105 111 105 6% Podkarpackie 216 214 217 235 253 229 18% Podlaskie 127 136 136 125 124 130 -9% Pomorskie 204 232 242 228 235 234 1% Śląskie 446 465 473 538 527 500 13% Świętokrzyskie 156 165 163 169 164 165 -1% Warmińsko-mazurskie 164 152 151 163 172 159 13% Wielkopolskie 339 360 348 379 363 362 1% Zachodniopomorskie 160 203 224 216 201 211 -1% B1 voivodships 207 217 222 231 232 225 6,6% Other voivodships 214 220 220 220 220 220 0,3% Region Difference in basis points (B1 – other voivodships) 6,4% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS 158 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Table A 14 Employment rate in the II quarter in the years 2000-2003 2000 2001 2002 2000-2003 mean* 2002/2000 % change POLSKA 56,6 55,2 52,0 54,6 -8% Dolnośląskie 51,1 49,0 48,9 49,7 -4% Kujawsko-pomorskie 53,0 52,8 50,6 52,1 -4% Lubelskie 63,5 60,7 56,7 60,2 -11% Lubuskie 50,2 50,8 47,0 49,3 -6% Łódzkie 57,5 54,8 53,4 55,2 -7% Małopolskie 60,8 62,2 55,9 59,6 -8% Mazowieckie 63,6 61,0 56,8 60,4 -11% Opolskie 57,3 54,9 50,6 54,2 -12% Podkarpackie 59,6 58,8 54,2 57,5 -9% Podlaskie 60,9 62,1 55,5 59,4 -9% Pomorskie 53,2 54,2 50,8 52,7 -5% Śląskie 49,4 49,2 47,6 48,7 -4% Świętokrzyskie 55,9 51,2 51,5 52,9 -8% Warmińsko-mazurskie 50,8 49,7 45,3 48,5 -11% Wielkopolskie 57,5 54,9 52,4 54,9 -9% Zachodniopomorskie 52,3 51,1 46,4 49,9 -11% B1 voivodships 56,4 55,0 51,3 54,2 -8,9% Other voivodships 56,4 55,7 52,1 54,7 -7,7% Region Difference in basis points (B1 – other voivodships) -1,2% * Geometric mean Source: Consultant’s own analyses based on data of Bank Danych Regionalnych GUS Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 159 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Service category Number of beneficiarie s planned in contracts Number of beneficiarie s quoted in final reports Efficiency planned in contracts % Achieved efficiency * % Achieved effectivenes s% Efficiency of services (all voivodships) No. of contracts Table A 15 Employment-related service 99 22115 21429 16 13 13 Training services 1130 22676 22691 48 32 32 Temporary employment services 602 7668 8125 24 31 38 Business Assistance Centres 148 18602 20523 8 9 10 Business Incubators 5 75 91 92 76 100 *Achieved efficiency is the number of employed beneficiaries of services to the number of all beneficiaries quoted in final reports submitted by service providers. Achieved effectiveness is the number of employed beneficiaries of services to the number of beneficiaries planned in contracts. Source: based of MIS-B1 data Figure A 1 Percentage of employed beneficiaries* of employment-related services 25 20 15 10 5 0 09/2002 03/2002 09/2003 12/2003 03/2004 06/2004 09/2004 03/2005 *persons who were provided complete services Source: based on MIS-B1 database 160 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Figure A 2 Percentage of employed beneficiaries* of training 45 40 35 30 25 20 15 10 5 0 09/2002 03/2002 09/2003 12/2003 03/2004 06/2004 09/2004 03/2005 * persons who were provided complete services Source: based on MIS-B1 database Figure A 3 Percentage of employed beneficiaries* of temporary employment services 40 35 30 25 20 15 10 5 0 03/2002 09/2003 12/2003 03/2004 06/2004 09/2004 03/2005 * persons who were provided complete services Source: based on MIS-B1 database Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 161 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Annex 2 Weaknesses and Threats Related to the Human Potential Activation according to SWOT Analysis based on Regional Development Strategies for the Years 2000-2004 (for Voivodships Participating in B1) VOIVODSHIP WEAKNESSES 1. Kujawsko-Pomorskie Education directions inadequate to [...] the labour market needs. Priority 1 – Education and human resources development Level of population’s qualifications inadequate to the labour market needs. Priority 2 – Creating conditions for entrepreneurship and efficient economic development Low level of population’s education. Priority 6 – Growth and differentiation of the economic base of region’s rural areas Low development potential of SMEs. THREATS Increased costs of education Low level [...] of population’s qualifications. Progressing impoverishment of population, particularly in rural areas. Low level of off-farming entrepreneurship development in rural areas (production, trade, services, transport, building sector) Little diversity of business activities in rural areas. Continuous low activeness and entrepreneurship, with the low level of rural population’s education. Free, unused and still growing labour force resources in rural areas. 2. Lubelskie Low level of farmers’ education. Progressing impoverishment of rural areas, lack of income from non-farm sources. Priority 2: Agriculture restructuring and rural areas development Low accessibility of non-farm jobs for rural population. Unfavourable demographic changes in rural areas. Low average wealth of rural inhabitants, irrespective of how much they are connected with farming. Unfavourable sociological phenomena, such as low level of education and of confidence in the positive significance of civic attitudes; lack of non-farm initiatives in farms. Priority 3: Employment and human resources development Little availability of local services for rural population. High unemployment. Still low effects of initiatives aimed at retraining. Little tendencies among entrepreneurs to develop their business. Little tendencies to launch one’s own small business – particularly among young people; lack of new entrepreneurs education and business incubation. 162 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. 3. Małopolskie High agrarian unemployment (high rate Outflow of qualified specialists. of persons employed in agriculture, lack of alternative sources of income for those working in agriculture). Farming business much fractionised and Low pace of transformations in rural producing too little goods. areas (country becoming a “museum piece”). Insufficient number of medium-sized businesses. 4. Podkarpackie Lack of people educated to run off-farm businesses. Priority 1: Multi-functional Lack of alternative sources of income, development of rural e.g. from agri-tourism areas 1A. Underdevelopment of human capital Lack of effective implementation of State’s structural policy in the field of agriculture and rural areas and of arms industry. Investors’ reluctance to locate business in low developed areas, strengthened by higher investment attractiveness of Małopolskie voivodship. 1B. Low entrepreneurship 5. Świętokrzyskie Serious discrepancy between high population rate of rural areas in the southern parts of the region and little opportunities for population migration to cities in this area. Unfavourable demographic tendencies, population’s ageing, outflow of young, active and well educated people. Disproportion in development [..] of small Low income per capita and unfavourable business between the northern and the structure of incomes (high proportion of southern parts of the region. disability pensions and old age pensions) having a negative effect on internal demand and local business development. 6. WarmińskoMazurskie Low development of entrepreneurship. Economic recession in the country. High unemployment and progressing economic and social degradation of the areas of former state agriculture and of border zone areas. Lack of Government’s stable policy supporting SEM sector. Population’s education irrelevant and inadequate to needs. External companies’ lack of interest in creating job placements in the voivodship. Lack of high-qualified human resources. Government programs concerning rural areas and agriculture not implemented. Lack of investor service centres and business information centres 7. Zachodniopomorskie Low occupational mobility of human resources – no system supporting entrepreneurship. Protracted recession in the Polish farming sector. Too slow modernisation and restructuring of rural areas and agriculture. Progressing economic and social marginalisation of some areas, expansion of structural unemployment and poverty. Lack of public [...] activity, development of demanding attitudes or passiveness and expectation. Source: Consultant’s own analysis based on: Development Program for Kujawsko-Pomorskie Voivodship until 2010, the Board of Kujawsko-Pomorskie Voivodship, Annex no 1 to resolution no 735/2001 of Sejmik (Local Parliament) dated 19 December 2001; Development Strategy for Lubelskie Voivodship, document abstract, Lublin, October, 2000; Development Strategy for Małopolskie Voivodship, Kraków, October 2000; Development Strategy for Podkarpackie Voivodship for the years 2000-2006 and up-date for the years 2004-2006, Rzeszów, April 2004; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 163 Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Development Strategy for Świętokrzyskie Voivodship, the Board of Świętokrzyskie Voivodship, Kielce, June 2000; Social and Economic Development Strategy for Warmińsko-Mazurskie Voivodship, " Development Strategy for Zachodniopomorskie Voivodship until the Year 2015", 23 October 2000. 164 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B1 Dominika Milczarek, Ph. D., Katarzyna Zawalińska, Ph. D. Annex 3 Description of Beneficiaries in a Breakdown by Services77 The employment-related services (job counselling) were provided to 21429 persons, 21175 out of this number having completed them. Women constituted the majority of participants (about 61%). School graduates accounted for some 5% of beneficiaries. The largest group of beneficiaries participated in these services in Świętokrzyskie voivodship (13103 persons were provided complete services) and the smallest – in WarmińskoMazurskie (2 persons). The average number of beneficiaries who were provided complete services amounted to 3025 per each of seven voivodships. The training services (development of qualifications) were provided to 22691 persons – this was more than in any other service provided under Sub-Component B1. 22174 beneficiaries have been provided complete training services. Men constituted the majority of people who participated in these services (about 55%). School graduates accounted for some 6% of these services beneficiaries. The average number of beneficiaries who were provided complete services amounted to 3168 per each of seven voivodships. The largest group of beneficiaries participated in these services in Warmińsko-Mazurskie (4820) and Świętokrzyskie voivodship (4644 persons) and the smallest – in ZachodnioPomorskie (1095 persons). The temporary employment services were provided to 8125 persons (6712 out of this number having completed the service). Men constituted the vast majority of people who participated in these services (about 84%). School graduates accounted for 1% of these services beneficiaries only. The average number of beneficiaries who were provided complete services amounted to 959 per each of seven voivodships. The largest group of beneficiaries participated in these services in Podkarpackie voivodship (1618) and the smallest – in Lubelskie (292 persons). 157 business assistance centres were provided support from the Project funds and they provided assistance to 21325 persons, 20252 of them having been provide a complete service. Women constituted the majority of this service beneficiaries (about 61%). School graduates accounted for more than 7% of beneficiaries here. The average number of beneficiaries who were provided complete services amounted to 2893 per each of seven voivodships. The largest group of beneficiaries participated in these services in Podkarpackie voivodship (6963) and the smallest – in Małopolskie (224 persons). 6 business incubators were set up under Sub-Component B1 in four voivodships: Kujawsko-Pomorskie, Małopolskie, Podkarpackie and Zachodniopomorskie. The services were provided to 107 persons, 69 out of them having completed it. Men constituted the vast majority of people who participated in these services (about 79%). There were no school graduates among the beneficiaries here. The largest group of beneficiaries participated in these services in Podkarpackie voivodship (24) and the smallest – in Kujawsko-Pomorskie (10 persons). 77 Based on MIS B1 data of the end of April 2005. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 165 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. 3 Sub-Component B2 Anna Wiłkomirska, Ph. D. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 167 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. 3.1 Top-down Analysis – Problems of Education in Rural Areas vs. Objectives and Outcomes of Sub-Component B2 3.1.1 Identification of Problems Sub-Component B2 Was Expected to Respond to Education problems are correlated with the social and economic situation of the whole country to a high degree. Their intensity and nature in rural areas and townships is a resultant of the relatively lower level of these areas’ economic and civilisation development. According to political ideas of democracy, reduction of social disparity, including educational disparity, should be important objectives of the social and economic transformation in Poland. What should be done in order to provide social advancement, definite improvement of living conditions, chances for active professional life bringing material satisfaction and making one see the point of doing of one does to disadvantaged groups? Studies made in the 90-ies78 enabled us to draw some general conclusions indicating that social barriers became established in the last decade of the XX century instead of being overcome. The following should be considered as the most significant barriers: • lack of social mobility – especially intergenerational – which is often a prerequisite of social advancement; this immobility is particularly common in rural areas, • unequal access to education; efforts to disseminate education at an increasingly higher level is accompanied by the decreasing proportion of students from workers’ families and particularly – from farmers’ families in Poland, • income disparities would remain a natural element of market economy, if they did not involve a strong and rapid pauperisation of large social groups concentrated in “poverty ghettos” – these are most often small places with a high unemployment rate and small, backward farms. Income disparities are not only a source of psychically favourable or destructive individual assessment of one’s position in the social structure, but they also cause serious social conflicts. The chance for overcoming these barriers is strongly related to the individual’s inherent status (the origin, family SES). The lower the family origin status, the less the chance for overcoming the barriers and achieving a social advancement and an individual success. The access to high quality education plays a crucial role in equalising the life start. In the nineties a clear acceleration of tendencies to make the access to secondary education common – the tendencies that had already been observed in the eighties, but the dynamics of disparities in access to university education is dramatic. In the years 19981999 the advantage of middle-class (intelligentsia) children with the university education over the children from farmers’ families was 51% to 5%79. The financial barrier – besides the gradually decreasing differences in educational aspirations of children and youths from various social environments – is an important factor: education is becoming more and more expensive, particularly at its higher stages. 78 Cichomski B., Morawski P., Polski Generalny Sondaż Społeczny. Instytut Studiów Społecznych, Warszawa 1995, Diagnoza Społeczna 2003, Domański H., Hierarchie i bariery społeczne w latach dziewięćdziesiątych, Instytut Spraw Publicznych, Warszawa 2000. 79 Ibidem. 168 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Moreover, there is a strong correlation between the level of education and the professional position in the market economy. The access to prestigious and first of all – stable professional positions increases as the level of education grows. Nevertheless, the inherent status (particularly – parents’ educational background) still remains a significant allocating force. According to studies made in the nineties, farmers are the social group characterised by a strong tendency to inheriting the family status (particularly – the father’s position)80. At the same time, income level analyses show that higher management staff, people with non-technical university background and business owners were the main beneficiaries of transformations in the nineties (CBOS, Jaźwińska 1999), while the income earned by farm workers and farm owners accounted for 68% of the national average at the most (Domański 2000). In 1999 more than 50% of farm workers and 25% of farm owners earned an income which was two times lower than the national average81. This analysis enables one to conclude, what steps should be taken up to provide access to education of the secondary and higher levels to disadvantaged groups. This would give them a chance to gain education enabling them to achieve a satisfying social and professional position and to take active part in counteracting marginalisation. The way leads through improving the quality of education in primary, middle and secondary schools, thus stimulating the growth of aspirations and real opportunities to extend the education path of children and youths from rural areas and townships. Results of Polish external exams for six-class pupils of primary schools and for middle school (gymnasium) last class students, as well as international studies of school achievements, such as OECD PISA or Civic Education Study, where Poland began to participate in the last decade, show that student’s social origin, school location (rural – urban) and school’s material equipment determine differences in students’ level of knowledge. Polish pupils from uneducated families, learning at schools situated in poor and neglected regions of the country, insufficiently equipped with didactic aids and computers, achieved results much below the international average82. At the information society age, students’ access to various media, including computers and the Internet used for a didactic purpose is a necessary condition of acquiring knowledge. In the year 2002, there were 65 computer rooms per 100 primary schools (without special schools): 86 in cities and 43 in rural areas. The situation was better in middle schools (gymnasia) – there were 93 computer rooms per 100 schools: 104 in cities and 82 in rural areas. It is evident that there are less computer rooms in rural schools than in cities. But, as these are schools with a smaller number of pupils, a more favourable rate of rooms per pupils is observed. In the year 2002, the number of computers in primary schools amounted to 615: 822 in cities and 408 in rural areas; 402 were with access to the Internet: 262 in cities and 140 in rural areas. The number of pupils per one computer was 40 in cities and 20 in rural areas. Middle schools (gymnasia) represented a better level of being equipped with computers – particularly in rural areas. 80 Koseła K. Młodym być. CBOS, 1995, Domański H. Hierarchie i bariery społeczne w latach dziewięćdziesiątych, Instytut Spraw Publicznych, Warszawa 2000. 81 Ibidem, Jażwińska E., Przedsiębiorcy jako grupa społeczna, Warszawa, 1999, Raporty CBOS 1999, 2000. 82 Śleszyński P. Ekonomiczne uwarunkowania wyników sprawdzianu szóstoklasistów i egzaminów gimnazjalnych w latach 2002-2004 , Expert report of the Ministry of National Education and Sports. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 169 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. It is neither quick nor simple to improve the quality of education. This process requires first of all: 1. Competent (educated and developing its skills) and devoted teachers who not only provide a high level of teaching, but develop students’ educational aspirations and confidence in educational success of the highest possible level as well, 2. A suitably equipped didactic base – a library full of books, multimedia didactic aids, computers and the Internet. 3. A wide and inexpensive education offer, activation of students’ extracurricular cognitive development, facilitated access and stimulation to individual exploration of knowledge, e.g. by the means of extracurricular activities, in the library, via the Internet. 4. Teaching methods that will arouse students’ cognition curiosity, ability of analytical problem solving, motivation to expand their knowledge. 5. Adequate funds, both for rewarding those who bear the great responsibility for educating future generations and for school functioning – renovations, equipment, transportation and meals for pupils. One might keep drawing up the list of conditions required to improve the quality of school work. Even those five listed above show, how difficult the task is, especially in rural areas, where the deficit of material means is most acute and where we have – statistically, on the scale of the entire country – fewest teachers with university degrees and poor didactic equipment in schools. The situation in the Polish education sector at the end of the nineties can not be analysed without considering changes that were introduced in an express mode behind the facade of one of the four great reforms. The key stage of education reform commenced in 1999 introduced the school system transformation, increased competence of local selfgovernment authorities in school management and maintenance and important modifications of the conception of school work and educational offer. The latter assumed changes in the methods of teachers’ work – responsibility for selecting or developing didactic and educational programs and rules of grading, as well as for adjusting them to the new organisational structure and curricular solutions, e.g. integrated teaching in classes 4–6. Teacher rather commonly agreed that the education system needs to be reformed, but this does not mean that they accepted the proposal and the mode in which it was implemented in 1999. This thesis is supported by outcomes of the research carried out by the Institute of Public Affairs83 and by the Social Evaluation conducted prior to RDP implementation. The way how the reform had been implemented was criticised first of all: without consultations with education-related circles, without preparing local communities, school directors and teachers and first of all – without the necessary funds. Education expenses represent a dominant part of rural municipalities’ (gmina) budgets. Teachers, in fear of losing their jobs, were sceptical about the school network consolidation, while parents were full of concern about sending children to schools distant from their homes. 83 Putkiewicz E., and others: Nauczyciele wobec reformy edukacji. Instytut Spraw Publicznych, Warszawa 1999. 170 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Moreover, the planned system of teachers’ professional promotion was also a matter of concern. Inhabitants of rural areas and personnel of primary schools were most reluctant to accept the reform. The criticism was correlated with the assessment of one’s own material standing which was lowest in smallest centres. One should also remember that the hasty mode of implementing even those changes that were accepted not only gave rise to the sense of threat, but paradoxically, it also contradicted the idea of decentralisation – one of the reform’s guidelines. Grassroots developments that had been commenced in support of schools in many gminas had to be suspended and the threat of closing schools that had been centres of social and cultural life in villages were regarded as an act of violence and disrespect for local needs. Nevertheless, most concerns applied to education’s most crucial problem, especially in rural areas: shortage of material means for meeting the challenges of the reform and of the current costs of school education as well. Decentralisation of the school system involved a significant increase of expenses from gmina budgets (gminas are granted subsidies from the State budget too). In 1990, the total outlays on education were as follows: Table 25. Education expenses of the State and of gminas (in total) in PLN million Years State budget’s expenses Education subsidies Gmina budgets’ expenses (including the subsidy) 1991 2813 - 717 1995 10 310 2195 5216 1999 19 471 16 551 25 161 2001 24 179 22 118 31 534 Source: the Main Statistical Office (GUS) The table reflects the growing share of local budgets in education financing. But the subsidy constitutes an almost 100% share of education in the entire budget of many poor gminas. One should note at the same time that the percentage of public expenditure for education accounted for 4.34% of GNP in 1991, 4.55% in 1993, only 4.14% in 1995 and 4.23% in 199984. Analyses of education in rural areas have been showing the need to improve its quality for years. This requires funds and involving the local environment in activities supporting development of their schools. Therefore, it seems very justifiable that Sub-Component B2 – Education was included in the Rural Development Project. One of many similar opinions expressed by persons involved in the project at the regional and local level will be an excellent summing-up of the analysis of education’s situation in rural areas based on scientific and statistical studies.85 84 Oświata i wychowanie w roku szkolnym 1999/2000, GUS, Warszawa 2001. Evaluation studies included interviews with directors of Voivodship Project Implementation Units, B2 regional consultants and representatives of gminas. 85 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 171 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. “...rural schools are very poor in general and very far from any, at least average standard of an average school. I do not even mean a well equipped school, with a good base. 90% or even 95% of these schools would never buy the things they bought under the Rural Development Project. I am speaking about didactic aids and computer software first of all here. Many schools would not even dream of such refurbishing. Or they would wait a very, very long time”. Evaluation of Sub-Component B2 Rationale The analysis presented above gives a basis for a positive assessment the decision to implement Sub-Component B2 under RDP. A vast majority of gminas would not make so many necessary refurbishment and modernisation works, would not buy so many didactic aids and/or software without the financial support. Fewer teachers would Participate in training – due to financial reasons and problems with physical access to centres running training courses for teachers. Consequently, it would be more difficult to achieve comparable education quality improvement indicators in rural areas. Both, representatives of VPIU and of gminas with whom evaluation interviews were conducted stressed very clearly the great opportunity for meeting renovation and investment-related needs, as well as for improvement of schools’ didactic base. Our respondents stressed that it was for the first time that such a comprehensive approach to education-related needs was adopted – the focus was not only on infrastructure development, but the need to develop teachers’ and school directors’ knowledge and to equip classrooms with didactic materials and software was taken into consideration too. It seems that the first seven voivodships were also selected reasonably. Except for Śląskie, these are poor regions, with a high unemployment level and inefficient farming. Gminas are not able to meet education’s financial needs on their own. On the other hand, it was the right decision to include all voivodships in Sub-Component B2 later on, since economic and civilisation differences between rural areas and large cities are very deep all over Poland, even in the richest Mazowieckie voivodship. It is certain that Sub-Component B2 met the criterion of being justifiable. 3.1.2 Evaluation of Sub-Component B2 Objectives and Activities Description of Objectives and Activities Activities relating to economic development of rural areas need to be accompanied by efforts to develop a strong education system, to provide such a level of knowledge, skills and ability to learn to school graduates from rural areas, which is necessary to continue education at the secondary and university level and to achieve the level of education and the professional position leading to a satisfactory material and social status. Such assumptions had been in the background of Sub-Component B2 objectives. 172 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Sub-component Activities Sub-component Project Objectives Objectives Figure 37. Sub-Component B2 objectives and activities in the context of RDP objectives Increasing off-farming employment Enhance the quality of teaching in rural schools Improve the quality of didactic and IT equipment in schools Supporting the process of strengthening self-government and regional development Improve the quality of graduates’ knowledge Didactic and computer training for teachers, school directors and education inspectors Improve the utilization of educational resources Refurbishing and equipping school facilities (consolidation) Assistance in building institutional potential necessary to absorb EU pre-accession and structural funds Enhance the education system’s effectiveness Refurbishing and modernization of school facilities Source: Consultant’s own analyses based on RDP documents According to RDP assumptions, the economic development of rural areas can be achieved through creating off-farm employment, developing human capital and infrastructure. The objectives and the activities of Sub-Component B2 are directly interrelated with all these conditions – with the human capital development most of all – also in the long run, i.e. development of education conditions conducive to the long education path of pupils from rural areas. According to the Project authors, the development of human capital consists in increasing and using the population’s professional potential, increasing the efficiency of the education system and the level of education, as well as increasing the effectiveness and the efficiency of local and regional administration units’ activities. The chance for human capital development lies in coherence of precisely designed activities of all SubComponents. Building the necessary infrastructure and supporting small business means building the labour market for future school graduates from rural areas. Enhancement of the education quality and better utilisation of individuals translate into reduction of unemployment. Increased competence of local self-government leaders implies more efficient functioning of all regional institutions and experience gained under the Project should be used when applying for EU funds. The general education-related objective of Sub-Component B2 was to increase the efficiency of the education system and the quality of graduates in target gminas by enhancing the quality of teaching in rural primary schools and gymnasia, and improving the utilisation of educational resources (Project Appraisal Document). Four main activities were financed within the frames of Sub-Component B2: 1. A program for improving the quality of teaching in primary schools and gymnasia in target areas through providing didactic training for teachers; 2. A program for improving the quality of school equipment through the provision of instructional material; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 173 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. 3. A program for supporting the introduction of computers for instructional purposes in rural gminas; 4. A program for improving school facilities (modernisation, refurbishing and renovation of primary schools and gymnasia) and education efficiency, through activities leading to the consolidation of the school network. Evaluation of objectives and activities relevance Objectives of the Project identify the education quality enhancement program and their respective tasks fit in the program in its various dimensions. The Project was implemented in the particular group of gminas, based on evaluation of the education system’s existing standing. Gminas were able to select from the above list of options a combination of developments that would help solve the main problems and face the challenges identified in the process of evaluation. As the evaluation of educationrelated needs carried out within the preparatory work showed, 84% of rural and ruralurban gminas meeting the eligibility criteria submitted applications for RDP co-financing that clearly exceeded the Project organisers’ expectations. The selection of activities was similarly adequate to the objectives. It is worth adding that gmina authorities concentrated clearly on material needs, with a strong depreciation of the need to develop the teaching staff by the means of training. This shows how vital the need to rehabilitate school facilities is and how strong the conviction that they are more urgent than training teachers. The scope of the Project and the means involved in it should be considered as significant. The requirement to make a financial contribution set local social forces in motion and mobilised them and this was a positive feature of the Project. Moreover, Project objectives and activities complied also with the national development policies. The regional development priorities provided for in the National Strategy for Regional Development for the years 2001-2005 include human resources development, as well as plan the support to be provided by the State to improve the school system’s infrastructure, to develop the information society and to counteract marginalisation. The objectives and the activities of the Project were connected with the educational policy of the State to a high degree – the education reform, i.e. implementation of the education management reform, curricula reform, professional promotion of teachers and school network consolidation (including organisation of pupils transportation) and in a later phase – including six years old children in the group subject to compulsory learning. It should be stated that Sub-Component B2 satisfied the criterion of objectives and activities relevance. Evaluation of participation The following candidates could apply for grants for infrastructure rehabilitation: 174 • rural gminas; • mixed rural-urban gminas, with the exclusion of tasks in city schools; • powiats (only for special schools), with the exclusion of tasks in city schools. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. The following candidates could apply for grants in the group of training and didactic tasks: • rural gminas; • urban gminas up to 15 thousand inhabitants; • mixed rural-urban gminas, with the exclusion of cities with the population above 15 thousand; • powiats, with the exclusion of cities with population above 15 thousand. Initially, the Sub-Component covered seven voivodships: Kujawsko-Pomorskie, Lubelskie, Małopolskie, Podkarpackie, Świętokrzyskie, Warmińsko-Mazurskie, Zachodniopomorskie, and since May 2003, the remaining nine voivodships have been included additionally in the rehabilitation and investment part (Phase I). Specific criteria used for evaluation of applications for didactic training are presented below: Table 26. Evaluation and scoring criteria (applications for didactic training) Ranking points Criterion Elements of evaluation Achievement of RDP objectives Gmina development strategy, selection of activities, importance of the project for gmina, education reform implementation. 25 Economic standing G indicator for the year 2001 25 Unemployment rate Preference for gminas with the highest indicator 15 Share of financial outlays in gmina budget Preference for gminas with the lowest rate of expenses per one pupil 15 Pupils/class rate as compared with the standard Primary schools: 18 (or more) pupils/class on the average – 12 points. Each one pupil less - 2 points deducted. 20 Gymnasium: 25 (or more) pupils/class on the average – 12 points. Each one pupil less - 2 points deducted. Special schools: 12 (or more) pupils/class on the average – 12 points. Each one pupil less - 2 points deducted. Source: RCS document The co-financing from RDP funds depended on the type of tasks to be performed and it might amount to the following proportions maximum: • 62% of the total cost of training teachers and school directors • 62% of the total cost of computer training • 59% of the total cost of purchasing didactic aids • 67% of the total cost of purchasing computer software. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 175 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. The criteria of co-financing were specified clearly. Preference was given to gminas with the low income, i.e. low outlays for education and high unemployment, and schools with a pre-defined pupil/class ratio close to the standard. Tasks to be co-financed could not apply to schools intended for liquidation under the school network consolidation process. Such eligibility criteria reflect the entire RDP’s objectives and the objectives of SubComponent B2. But in case of poorest gminas, 30-40% of costs that had to be contributed by gminas might be a difficult barrier to overcome and for some of them this was a significant limitation of their ability to act which was inadequate to needs. On the other hand, active participation in co-financing the developments on gmina’s own territory is an obvious criterion of responsibility and commitment to development of the local environment. The participation was regulated by the eligibility criteria, by gminas’ financial potential (own contribution) and by the funds available under the Project. Most serious needs were identified in the field of renovation and investment tasks. The applications for rehabilitation tasks exceeded the available co-financing 6 times as early as in 2001. The demand for financial support in the area of Sub-Component B2 activities varied: in case of didactic training it was satisfied almost entirely, sine it turned out to be smaller that expected. Besides, gminas were not always interested in or had sufficient funds for implementing the idea of combining participation in both types of training and purchasing both the didactic aids and the software as well. One should remember that qualification criteria related to the number of students in classes often eliminated small schools that needed support very much (this was stressed by many representatives of VPIUs and of gminas). On the other hand, gminas’ scarce budgets set restrictions of the number of applications too. Evaluation of participation, even if not measured in terms of the number of beneficiaries to the number of potential beneficiaries, but in terms of participation of those who needed it in fact, i.e. by the number of applications rejected to those accepted, will to a high degree result from the immense needs of schools situated in rural areas – the needs concerning refurbishment and modernisation and the need to equip schools with didactic aids, including computer hardware and software, with the limited financial potential determining gmina’s own contribution. Therefore, considering the budget of the Sub-Component and the scope of the Project, one should regard the level of attendance as satisfactory. On the other hand, the scale of schools’ refurbishment-related needs and the criteria limiting participation of small schools were the main factors limiting the possibility to satisfy the needs reported by schools. Our respondents involved in project co-ordination and the level of voivodships and gminas also indicated these limitations. When asked about the barriers to access to the program (What were the most significant barriers to implementation of tasks designed under Sub-Component B2?), many respondents gave answers similar to the one quoted here: “The size of the school is the barrier, in my opinion. The co-financing amount is another barrier. It should be higher. – “50 to 50 at least”. “Depending on gmina’s contribution – the higher the contribution, the bigger the co-financing” (gmina representative). 176 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. A different opinion about co-financing was presented by another respondents, who noted unnecessary – in his opinion – restrictions concerning e.g. the number of pupils in school: “Namely this Sub-Component we participated in, was a training and didactic one, while we were not provided any support in case of the infrastructure-related application. As far as the infrastructure development was concerned, we applied for funds for gymnasium in Z... and in this case the criteria were unfavourable for the rural environment. The number of pupils in schools was one of the main criteria and in case of such environments and rural gminas as S..., we are losing with larger environments at the very start. With such scoring criteria, townships up to 5 thousand inhabitants had an enormous advantage over us. As far as gmina’s own contribution to projects was concerned, richer gminas representing a higher level of civilisation development had more opportunities, more room to begin with. In case of small gminas, providing an own contribution – which might seem not very high, ranging from several dozen or several hundreds of thousand – is a very serious load for them and considering that some gminas are in debts, it is impossible for them to participate in any projects that can be implemented in rural gminas. This means that the economic barrier may be a hindrance to reaching for money and it may affect the chances for making the starting point of country and city children equal.” (gmina representative). But not all respondents showed a similar social sensitivity: “These criteria were as they were, because this program is for rural areas – this was stressed many times that the idea was to level the differences out and in fact, gminas in a worse financial situation were always given preference, i.e. those where the income per inhabitant was lower and where the unemployment rate was higher. This was taken into account. Not everyone liked this, because sometimes those richer and thriftier lost. But this is a matter of assessment” (gmina representative). Some respondents thought that the program should be extended over secondary (postgymnasium) schools, as they are disappearing from rural areas, and over non-public schools (although opinions were divided in this matter). It is worth remembering at this point, what was also mentioned by some of the respondents – that local associations take over schools more and more often and that they need financial support very much. To sum it up, one should note that the funds available in the Sub-Component were used in a manner ensuring the highest possible participation of eligible applicants. The number of beneficiaries interested in Sub-Component’s tasks was limited by criteria disadvantaging small schools and by the low level of their own funds. Targeting evaluation The objectives and the activities of Sub-Component B2 fit in the State’s declarative policy in the field of education and it was coherent with the assumptions of the reform commenced in 1999. The selection of eligibility criteria and the rehabilitation and modernisation tasks supported the project of consolidation (at the first stage of the rehabilitation program gminas had to attach data form strategic plans for education to their applications). Training tasks facilitated the introduction of requirements of the curricula reform and enabled professional development required by the new system of teachers’ professional promotion. No other assistance project supported the needs of schools and education to such a high degree, particularly in rural areas. The project can be considered as an element of difficult initiatives aimed at providing children from rural areas equal chances for education. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 177 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. 3.1.3 Situation before and after Implementation of Sub-Component B2, Outcomes of the Sub-Component86 By the end of December 2004, the number of contracts for renovation of schools amounted to 1,799 and in case of equipping and renovating school common rooms – to 955. Additionally, 314 contracts for equipment of pre-school education classrooms were signed. All in all, 1271 schools were renovated by the end of 2004 for the total amount of PLN 171 million. 2974 various rooms were renovated in schools. Within the frames of the school network consolidation, 187 common rooms were renovated for the amount of PLN 8.6 million. 425 common rooms were equipped and the equipment value amounted to PLN 5.7 million. Besides, 212 pre-school education classrooms were equipped for the amount of PLN 1.3 million. These figures increased in 2005 – by the end of March 2005, 1885 schools, 4126 classrooms and 259 common rooms were renovated, 682 common rooms and 491 pre-school education classrooms were equipped in total. Some of the contracts are still being executed and accounts are being cleared. It is worth stressing that the outcomes exceeded the plans as early as at the end of March 2003. By the end of October 2004, the cycle of didactic training for teachers and school directors had been completed. The training provider – STO – conducted training for 8979 teachers (without training in information technology), including 748 school directors. It should be noticed that fewer teachers were trained than assumed - around 97% of the planned number of 9240 trainees. The contract with STO provided for the amount of PLN 10.2 million. By the end of November 2004, training courses in information technology delivered by the Consortium of Polish Training Companies were completed. The courses were attended by 7716 participants, including 1012 IT teachers, network administrators and managers of computer rooms. Moreover, 1300 school directors and education specialists were trained in territorial self-government units and 5404 teachers of other subjects. Not all training courses that had been planned were delivered. The number of training beneficiaries accounts for some 94% of the planned performance – initially 8209 potential participants had been reported. Gminas purchased 711 software packages for around PLN 4 million. The value of purchased software was higher than planned. By the end of October 2004 deliveries of instructional materials and school equipment were effected. In total, schools ordered 746 packages of didactic aids (about 30,000 various items). The value of purchased items amounted to PLN 22.63 million – more than planned. Since April 2004 the portal named “Scholaris” – the Internet Centre of Educational Resources has been active under the supervision of the Ministry of Education and Sports. The portal will continue its functioning, its cost to be covered from structural funds. In the year 2005 the www site references indicator was three times as high as planned. On the other hand, the number of new internet addresses (2171) and of www sites created by 86 All information about performance of Sub-Component B2 tasks is quoted after MIS Monthly Reports, and data were submitted to the Project Co-ordination Unit by Voivodship Project Implementation Units. 178 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. teachers and students (1854) was much lower than planned (5000). First issues of monthly magazines titled Nowe w szkole (“The New at School”) and Kierowanie szkołą (“Managing a School”). The portal is gradually becoming more and more popular. Common Rooms Renovation and Furnishing; School Facilities and Classrooms Renovation Applications for co-financing of rehabilitation tasks in schools and common rooms in 7 voivodships were mainly submitted in the year 2001. Most of these tasks were completed as soon as in 2002. In May 2003 a decision was made to allocate an additional amount of money for renovation and equipment in schools (with a particular focus on pre-school classrooms) and common rooms from the Project reserve fund, as follows: EUR 1 million for 7 voivodships where Sub-Component B2 – Education had been implemented since the beginning of the Project and EUR 6 million for 9 voivodships that had not participated in this Sub-Component before. The deadline for this task completion had been planned for December 2004. Total number of equipped common rooms Total cost of renovated common rooms (PLN)* Total cost of common rooms equipment (PLN)** 111 $261 m N/A 185 218 119 140 259 463 682 (+491 pre-school classrooms) 219 $2,54 m (cost of renovated common rooms + $2.05 m (cost of equipment) + $1.21 m (pre-school classrooms) 2005) 118 Difference (31/03/ 88 31/03/2005 52 12/31/2004 12/31/2003 Total number of renovated common rooms 30/06/2004 Indicator 30/09/2003 Results of renovating and equipping school common rooms Expected outcomes Table 27. +157 +571 +491 (preschool classrooms) - $0,5 m (common rooms renovations) N/A * Total and unit gross cost of renovated common rooms (renovations and equipment – gross amounts). ** Total and unit gross cost of equipping common rooms (PLN). Source: MIS Monthly Reports and RDP Monitoring Table (B2) Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 179 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Figure 38. Common rooms renovation and equipment – comparison of expected and achieved results 800 700 600 500 400 300 200 100 0 renovated common rooms equipped common rooms expected results equipped pre-school classrooms achieved results Source: MIS Monthly Reports and RDP Monitoring Table (B2) The proportion of expected results to achieved results is impressive and similar is the picture of the balance of renovation tasks in schools. 1689 1885 365 446 2005) + 1785 +$51.64 m $59 m 328 Difference (31/03/ 2005 31/03/ 2004 348 196 12/31/ Number of renovated teaching rooms ** 2004 $7,36 mln 201 30/06/ Total gross cost of school renovations (PLN) 140 2003 100 12/31/ Total number of renovated schools 2003 Indicator 30/09/ Outcomes of renovation tasks in schools Expected results Table 28. 3680 4126 +3778 **teaching rooms: classrooms, laboratories and gymnasia Source: MIS reports 180 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Figure 39. Renovation of school facilities – comparison of expected and achieved results 4500 4000 3500 3000 2500 2000 1500 1000 500 0 Renovated schools renovated facilities Expected results Achieved results Source: MIS reports Professional development of teachers and school directors Training was less interesting to beneficiaries than rehabilitation activities. Additionally, the number of candidates could be reduced by protracted bidding procedures and the short holiday period of training. Table 29. Professional development of teachers and school directors – outcomes Difference Indicator Expected outcomes 30/09/2004 30/12/2004 31/03/2005 School directors trained in management 600 748 748 748 +148 10 000 8 202 29 8231 -1769 4 000 4 717 5404 +1404 958 1012 +212 1 070 1300 +900 Teachers trained didactic issues in Teachers trained in IT IT teachers trained Managers of computer rooms trained School directors and JST specialists participating in IT training 400 31/03/2005 400 400 Source: MIS reports The cost of training one person (including the teacher’s financial contribution, but without costs of travelling) amounted to PLN 1100.29 in case of didactic training and PLN 760.77 in case of IT training. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 181 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Purchased educational materials and computer software The idea of Sub-Component B2 organisers had been to offer schools and teachers from rural areas participation in didactic and IT training and at the same time the opportunity to purchase didactic aids and software intended to improve the quality of teaching. The idea was not realised entirely, but the purchasing indicators were higher than planned. Table 30. Purchased didactic aids Indicator Expected results 30/09/2004 30/12/2004 31/03/2005 Purchased packages of didactic aids 600 746 746 746 Difference 31/03/2005 +146 Value of purchased packages of didactic aids (PLN) $4.39 Purchased software 400 Value of purchased software (PLN) $0,9 m $5,3 711 711 +$0.91 711 1m +311 +$0.1 m Number of schools that received didactic aids 1863 1863 1863 1863 tbd Number of schools that received software 1720 1720 1720 1720 tbd Source: MIS reports Deliveries of educational materials took place in September and October 2004. Figure 40. Purchased didactic aids and software – comparison of expected and achieved outcomes 800 700 600 500 400 300 200 100 0 didactic aids Expected results software Achieved results Source: MIS reports The Internet Centre of Education Resources It had been an ambitious idea to create the Internet portal and to encourage education personnel from rural areas to co-operate. But most probably creating www sites of 182 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. schools, teachers or pupils even, requires a better technical base and overcoming motivation barriers. 30/09/2004 30/12/2004 31/03/2005 Difference 31/03/2005 Use of “Scholaris” Internet Portal Expected results Table 31. 200 000 137 000 445.332 700 000 +500.000 Number of www addresses created by schools and students 5 000 1 364 2171 2171 -2829 Number of www sites created by schools and students 5 000 250 1854 1854 - 3146 Indicator Number of references to the www site Source: MIS reports Figure 41. Use of Scholaris Internet Portal – comparison of expected and achieved outcomes 6000 5000 4000 3000 2000 1000 0 Internet addresses Expected results www sites achieved results Source: Consultant’s own analyses Summary of outcomes The scope of the Project should be considered as extensive. 259 common rooms were renovated and equipped, and this was important with regard to the education reform implementation (mainly the problem of transporting children to schools). This figure may be not very impressive, but one should remember about gminas’ more urgent rehabilitation needs. Necessary renovation was effected in some 300 schools (in the first 7 voivodships) and this resulted in reduced maintenance costs in many cases. Didactic training was provided to more than 8000 teachers and 600 school directors; IT training was provided to 2300 education employees. Schools were equipped with 700 packages of didactic materials, the cost of which exceeded PLN 22 million. These included complete sets of equipment for computer rooms, for example. Moreover, local self-governments ordered more than 700 software packages. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 183 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Expected economic changes According to the assumed objectives, the changes resulting from the education SubComponent are of a delayed nature rather. Rehabilitation of school facilities may bring results relatively soon – on the one hand in the form of reduced school maintenance costs, e.g. less expensive heating, but on the other hand – in the form of increased costs of starting additional activities, e.g. in newly equipped laboratories of new sports facilities. It is difficult to estimate the direction of financial decisions made by gminas and school directors. One should not expect any other immediate and quick changes of economic indicators. Any economic effects resulting from the enhanced quality of education will be visible in a dozen of years or so and they will be correlated with many other social and economic factors to a high degree, therefore they will be difficult to separate and quantify. Evaluation of effectiveness The expected outcomes were exceeded evidently, particularly in the rehabilitation phase. On the other hand, the performance indicators were lower than expected in case of didactic training for teachers, IT training and use of Scholaris Internet portal. One may indicate at least two significant reasons of deficit in this field. It seems that rehabilitationrelated needs of gminas in the areas covered by the Project had been so immense and so difficult to satisfy with such a serious shortage of funds, that they became beneficiaries’ priority. Besides, training services were provided in the second phase of activities. Too much time had passed between acceptance of applications and the phase of implementation and the scope of needs had changed. As far as Scholaris portal is concerned, one should reserve the final judgement, considering how short the time of its operation has been (with the holiday period included) and bearing in mind constant problems with an easy access to the Internet in rural areas. The very chance to visit an educational Internet portal is an opportunity to update knowledge and to share professional experience for the teaching staff. Having analysed opinions expressed by the beneficiaries in interviews, one should discuss a few issues in detail. The access to information about Sub-Component B2 was assessed very well by representatives of Voivodship Project Implementation Units (VPIU), representatives of gminas and school directors. A vast majority of the respondents had no problems with obtaining information they needed. Applications were found easy to fill, particularly when compared with other assistance programs. No serious problems were encountered during the implementation of project activities. Persons responsible for the project in particular gminas praised regional co-ordinators’ professionalism, readiness to provide any assistance that was needed and short time of response. Training courses for gminas’ representatives and Internet information packages were given high marks. Respondents from the VPIU were more critical to the Project Co-ordination Unit. They complained of frequent rotation of officers responsible for particular tasks, protracted procedures, unclear distribution of competence between PCU and the Ministry of Finance. All respondents expressed their gratitude for the very possibility to obtain help, but the cofinancing was assessed as too low in case of rehabilitation tasks. Moreover, criticism was expressed with regard to the relatively long time of waiting for funds after the invoice for works was submitted. But in general, organisation of the rehabilitation phase was assessed much better than organisation of training, although there was definitely too little time for renovation works at the second stage and gminas signed contracts with unrealistic dates of completion, especially that some of the works had to be performed 184 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. during the school year, i.e. only in the afternoons, when students were not at school. In some cases, bidding procedures were protracred as a result of gminas’ tardiness, bur contracts for immediate performance of works were signed anywa, as renovation was very important for the beneficiaries. But the beneficiaries were most critical of the long time of waiting for activities after the application for training had been accepted. This caused many administration and financial problems (changes in gminas’ budget plans) and on the other hand, after such a long time the candidates were not interested in training any longer. Below follows one of many similar opinions: „...as far as effects of Sub-Component B2 are concerned, my assessment of training is very high. All the materials that were provided, the software – this was all on a very high professional level. But organisation was more than bad. Bidding procedures conducted by the PCU took from 6 months to 2.5 years, and this is simply ridiculous. Thereafter – the performance itself, in technical terms: in one case we signed 10 annexes to the contract for didactic training with one of gminas” (regional consultant). The procedures, that involved not only PCU representatives, but also representatives of all voivodships and – in case of Sub-Component B2 – representatives of MONES, took relatively much time, due to the rules of creating bidding documentation, verification and and approval thereof. This resulted in delays that caused problems to beneficiaries. In the opinion of VPIU representatives, STO was more successful in organisation of didactic training than the consortium delivering IT training. Problems with enrolment were reported, as the opinion quoted below shows: ”There were some problems in case of IT training. At a certain point, there was a huge hold-up resulting from the fact that the company which had won the bidding, the Consortium of Polish Education Companies, was not prepared to collect information, who, where and in which training should participate. This is such a down-to-earth issue, but it is very important. The company did not take into consideration that there was no access to the Internet in gminas, no knowledge that would enable people to log in the system, select a course using passwords and other procedures. Maybe this was interesting, but I don’t know if this was needed – maybe it was. An all in all, there were masses of phone calls (...) how to do this” (representative of a VPIU”. The Consortium, when proposing the form enrolment – this being the first educational task – had not envisaged the scale of difficulties of both technical and competencerelated nature. Irrespective of the beneficiaries’ very good general opinion on Sub-Component B2, the shortcomings described above contributed significantly to the fact that not all available funds were spent. In general, the effectiveness of the Project should be assessed very high, particularly in the area of rehabilitation tasks and providing didactic aids to schools. The needs of gminas situated on the territories covered by the Project are immense. The Project enabled many of them to conduct the necessary renovation leading to improvement of the school infrastructure, but also to reduction and re-distribution of future costs, e.g. for thermal insulation of buildings. Consequently, it was possible to satisfy a significant part of material needs. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 185 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Utility evaluation The rehabilitation activities, the supply of equipment, didactic aids and software, the training services will make it possible to meet the requirements of the education reform better and, what is more important, they give a chance for enhancing the quality of education. New didactic equipment in schools and the use of active teaching methods popularised in the course of training may result in increased motivation to learn and to increase students’ educational aspirations. Moreover, participation in didactic training is one of conditions of teachers’ professional promotion. It is not possible to carry out any thorough evaluation of qualitative changes in such a short time, but it will be possible within the nearest years through monitoring the achievements of students from rural areas and their education careers. Although the effects of using Scholaris Internet portal were lower than expected, there is a chance that it will become an important forum of education-related information, exchange of teaching experience, writing efforts and presentation of action research outcomes. The evaluation of Sub-Component B2 utility should be supplemented with data obtained from potential beneficiaries, i.e. education personnel from gminas that did not participate in RDP. A half of school directors covered by the survey heard about the Project. 1/5 of gminas where the respondents are employed applied for co-financing. Certainly, rehabilitation tasks were most interesting to them, IT training taking the second place. In most cases, schools did not become RDP beneficiaries, as their gminas were not able or did not want to provide the own contribution or the information about the Project was received too late. Some schools were not eligible, e.g. they were too small, run by associations or were not interested in training. School directors, if offered the opportunity to apply to the Project today, would be very happy to receive co-financing for renovation, to but equipment for the common room, software and instructional materials. It is worth adding that 1/3 of schools managed by our respondents have already benefited from assistance programs, mainly being provided a base of computers. Maybe this is why 76% have computers, although only 1/5 have a separate computer room. Almost all schools are connected to the Internet. ¾ have instructional software too. About a half of school directors participating in the survey stated that the it was not often used. But in many cases, the equipment does not meet the parameters required by modern software and premises are insufficient for computer technologies to be used on a common basis. 70% of schools that did not participate in the Project have their sports facilities (gymnasia) and 80% - common rooms for students. 18% of respondents described the technical condition of the school building as very good, 48% as rather good and 15% as bad. Almost all school directors from this group were planning some – necessary in their opinion – renovation works (most often – replacing windows, floors, roof, renovating the boiler room). More than a half intended also to equip common rooms and 1/3 – to equip pre-school classrooms. Other urgent needs included purchasing didactic software (45%), computers (30%) and instructional materials (36%). According to data obtained from school directors that did not participate in RDP, although their schools handled the situation well in the fields covered by the Project support, most of them would be happy to obtain co-financing in order to meet some urgent rehabilitation needs and to purchase better didactic aids. 186 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. 3.2 Bottom-up Analysis – Project Effects Evaluation 3.2.1 Identification and Measuring of Sub-Component B2 Cost All tasks performed under Sub-Component B2 were financed from a few sources: Table 32. Types of funds used in Sub-Component B2 Funds Amount Loan in EUR 32 067 802* Loan in PLN 133 892 419 Budget 880 386 Own contribution 186 012 941 * Rate of exchange 1 EUR = PLN 3.95 has been accepted. Source: Consultant’s own analyses By 17.05.2005, 96.93% of the funds provided under the World Bank’s loan were absorbed. Efficiency evaluation Refurbishing and investment tasks were performed at the cost below the plan and as a result of this schools from more distant places in the ranking could be included in the Project. Costs of training and prices of didactic materials and software were not higher than those quoted on the market. 3.2.2 Identifying and Measuring Products and Effects of Sub-Component B2 Significant quantitative effects were achieved in Sub-Component B2 and they exceeded the planned performance considerably in the field of rehabilitation tasks. On the other hand, in case of didactic and IT training for school directors and teachers, the indicators were below the expectations, but relatively high anyway. Moreover, the performance was beyond the expectations in case of didactic materials and software purchasing. But to be able to regard the Sub-Component activities as effective, one needs to carry out a qualitative evaluation as well. The basis for this evaluation was provided by the results of survey conducted on representative samples of beneficiaries of all Sub-Component activities. Refurbishing and modernisation of schools, refurbishing and equipping common rooms Refurbishing and investment tasks in school facilities were considered to be SubComponent’s most desirable activities. In some voivodships, 1/10 of applications was qualified for co-financing. The fact that additional funds were allocated for this purpose in the second phase of the Project implementation should be regarded as a great success. The needs were urgent – in 1/5 of modernised schools to which interviewers came, renovation had been recommended by Sanepid (the sanitarian and epidemiological inspection). Representatives of gminas and school directors expressed positive opinions on bidding procedures, with one exception – the fact of inviting many firms that had not been interested in small assignments was regarded as a hinder. Representatives of poorest Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 187 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. gminas indicated even the unnecessary costs of sending letters (envelopes, post stamps). The vast majority of beneficiaries assessed the workmanship of refurbishing very high, although some respondents from gminas indicated that the need to reduce costs as much as possible (the basic bidding criterion) implied using building materials of a lowest quality for example. The following immediate benefits resulted from refurbishing tasks: • improved conditions of teaching, • improved safety of users, • improved appearance of buildings and rooms, • economies in heating energy expenditure. Comments on the rehabilitation activities organisation Many of the school directors covered by the survey (40%) did not make any critical comments. The others made a plea for improving the co-financing timing and for increasing the co-financing, as the needs were immense. Training for school directors In the year 1999, when introducing the education reform, the Ministry of National Education and Sports decreed that all school directors needed to acquire management competence. One of training courses planned under Sub-Component B2 – School Management during and after the Education Reform was intended for school directors. Besides the very requirement to develop qualifications (most of them had already attended a training like this), school directors emphasised the opportunity to improve their management skills, this enabling them to make the school work more efficient in terms of administration and education as well. A vast majority of this training participants who were interviewed – 86% - declared their participation themselves; further 9% were appointed by gminas. School directors expressed very high opinions about the quality of training. Using the 5-degree scale, the participants evaluated a number of elements. Their answers are presented in the table and on the graph below. Table 33. Marks given to trainers – in %, N = 296 Evaluation criteria Positive marks Neutral and negative marks (including “hard to say”) Rather good Very good Knowledge of the issue 14.5 82.2 2.7 Ability to present information clearly 14.1 78.8 7.1 Commitment to training 11.2 83.3 5.5 Source: Consultant’s own analysis 188 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. The material obtained from questionnaires shows that training participants marked trainer’s competence very high, particularly knowledge and commitment. Similarly high marks were given to training proper. Figure 42. Average evaluation of training for school directors (scale from 1 to 5, where 1 stands for very bad and 5 for very good) 5 4 3 2 Location Training materials Duration of training Date of training Waiting time Knowledge development New skills Usefulness for school work 0 Facility of enrollment 1 Source: Consultant’s own analyses Relatively lower marks were given for the long waiting for training after one had been qualified for it. This had been as long as nearly two years. Consequently, professional plans and competence of many candidates had changed. In case of school directors, the problem was even more serious, as since 1999 the law imposed an obligation to attend a course on them. The co-financing procedure did not allow for reduction of the number of candidates – the only acceptable change was to replace one candidate with another one and this caused a sudden need to look for new volunteers. In practice, all – 99% of the school directors covered by the survey – had participated in other school management courses earlier. During the course, the directors developed (according to 1/3 of the respondents, the course did not offer any new skills) or acquired new knowledge and skills in the field of exercising pedagogic supervision and measuring the quality of school work. The participants considered the competence in the field of school management and preparing school development plans to be important too. Moreover, the beneficiaries stressed the development of interpersonal skills, communication skills and team work leadership. Besides knowledge development, 1/3 of training participants stressed the significance of meeting many other school directors, exchanging experience with them and establishing co-operation. Only 8% of the respondents thought that participation in training did not bring any benefits at all. A vast majority of training participants – 91% - makes use of the skills acquired during the training. Most often – when preparing school development plans (the legal obligation), keeping school documentation (more and more extensive due to various ministerial Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 189 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. instructions), organising and evaluating school work and running staff meetings. Only 9% of directors began using computer more intensely for managing their school. Didactic training for teachers Both representatives of gminas and teachers, when asked to explain, why they had chosen namely this task, stressed two motives mot frequently: the requirement to improve qualifications and the opportunity to attend training at a lower price (gminas) and at no charge at all in practice (most teachers). Moreover, the need of continuous improvement of teaching standards was emphasised too. Fewer respondents mentioned the need to make their knowledge more complete and this also leads to improvement of professional qualifications, certainly. The access to professional development is much more difficult for teachers working in rural areas than for those from larger cities, but many of them overcome the difficulties effectively.87 One should stress that most of the teachers participating in the survey – participants of RDP training courses – had adopted an active approach to self-development. Only 1/5 of them did participate in any training within the last two years. 36% attended short forms only (up to 20 hours). Nearly 2/3 of teachers covered by the interview had chosen the training on their own. A relatively large group consisted of those who had been delegated by their directors. More than 90% attended the training during their paid holiday (summer holidays) and some of them expressed a dissatisfaction with this fact. Most teachers stated that they had not had to satisfy any special conditions in order to participate in training. Very few mentioned that they had lost a part of their holiday, some mentioned the necessity to teach a concrete discipline and some reminded of the requirement to pay about 200 PLN. Społeczne Towarzystwo Oświatowe (STO, the Social Education Association) was selected in the bidding procedure to provide didactic training. Teachers were offered 10 types of training, including the one for school directors which has already been discussed above. The courses were 70 hours long. The subjects (without the training for directors) are presented in the table below. 87 Murawska B., and others. Wsparcie rozwoju zawodowego, a potrzeby nauczycieli w tym zakresie, Instytut Spraw Publicznych, Warszawa 2005. 190 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Table 34. Training topics and attendance Training title Attendance Application of activating teaching methods 1815 Development of educational skills 1808 New rules of grading students at all levels of education 1185 Integrated teaching in classes I-III 1073 Developing curricula at the class level 900 Natural science teaching in classes IV-VI 431 Educational development in the discipline of civic education and history in classes IV – VI 404 Elementary pre-school education 312 Teaching in integrated classes 303 Total 8231 Source: EPRD report It shows that two courses were most popular - Application of activating teaching methods and Development of educational skills. Teachers’ choice was very reasonable. Increasing students’ motivation to learn and be active during lessons, as well as teachers’ helplessness to handle the growing educational problems are two greatest problems of school today. The last four courses listed in the table were relatively least popular. The first two of them are of a supplementary nature and were intended for teachers of concrete subjects, while those more popular were intended for all teachers. The last two courses were interesting to a narrow group of receivers only. Very few children attend kindergartens in rural areas in Poland, and this is why the group of teachers is not numerous here. Similarly, the number of integration schools and classes is relatively small. It is worth stressing however that these least popular courses were given the highest marks by trainees. The analysis covered programs of training courses and training materials that were given to trainees. All of them should be assessed positively. They were interesting, comprehensive and up-to-date. Most of trainees filling questionnaires indicated that the range of offer was rather wide – 56.7% or very wide – 27.2%. In most cases, teachers made a decision to attend some particular training themselves – 61.1%. 1/3 of teachers were appointed by directors of their schools and 3% replaced other candidates. The table below shows how trainees evaluated training adequacy to the needs of schools and teachers. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 191 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Table 35. Evaluation of didactic training adequacy to beneficiaries’ needs in %*, N=573 Positive marks Rather high degree Very high degree Negative marks and no mark at all (not at all, very low and rather low, hard to say) Teaching staff in rural areas 59.9 27.4 12.7 Teaching staff in the school the trainee works at 59.9 26.4 13.7 The trainee 47.5 36.3 16.2 To what degree does the training meet needs of: *Data apply to all didactic courses in total, there were no statistically significant differences between evaluation of particular training courses. Source: Consultant’s own analyses Only some 10% of the respondents gave negative marks when evaluating the degree of didactic training adequacy to the needs of teachers and schools in rural areas.. Similarly, trainers and the contents of training were given positive marks. Specific results are presented in the table below and on the graph. Table 36. Evaluation of trainers * in %, N=573 Positive marks Evaluation criteria Neutral and negative marks (and “hard to say”) Rather good Very good Knowledge of the issue 12.8 83.2 4.0 Ability to present information clearly 13.8 81.3 4.9 Commitment to training 9.8 85.1 5.1 * Data apply to all didactic courses in total, there were no statistically significant differences between evaluation of particular training courses. Source: Consultant’s own analyses There is no doubt that a vast majority of trainees were of a good or very good opinion about the competence and the commitment of trainers. 192 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Figure 43. Evaluation of didactic training (on the scale from 1 to 5, where 1 stands for very bad and 5 for very good) 5 4 3 2 Location Training materials Training duration Waiting time Facility of enrollment Knowledge development New skills Usefulness for school work 0 Date of training 1 Source: Consultant’s own analyses The teachers gave highest marks for usefulness of new information for their school work, for facility of enrolment and for training materials provided to them. The time of waiting for the course and dates of courses were given lowest marks – just as in case of other training courses. Participants of didactic training, when listing the skills they developed or acquired during training activities, emphasised psychological competence – mainly interpersonal and communication skills – and methodology of teaching. Participants of most popular courses stressed the fact of acquiring knowledge about new methods and increasing their educational competence. Those who attended other courses, mentioned the skill to grade students or to develop curricula. These skills seemed important to them, as universities rarely provide such competence to future teachers. Besides development of professional competence, teachers also referred to “strengthening of personality” in the field of selfknowledge and self-esteem. The trainees were also satisfied with the opportunity to meet and to discuss in a group of people with similar problems. 11% of trainees were of the opinion that they had not learn anything during the course. A very high percentage of the respondents – 92% – stated that they used the newly acquired skills in their everyday school work. This applies most frequently to: • activating teaching methods, • new methods of grading, • new educational solutions, e.g. contacts with students, • developing new curricula, • co-operation with parents, • co-operation of the school’s team of teachers. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 193 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Moreover, 2/3 of the trainees declared that they shared the knowledge acquired during the training with their colleagues from school, including – what was important – the interns. Some teachers set up self-learning teams in their schools, teams developing curricula, educational teams. Some others presented their newly acquired skills during special staff meetings or conferences, teaching others what they had learned themselves. Such a cascade effects of training should be considered a great success. Comments on organisation of didactic training The trainees were also asked to formulate their suggestions concerning organisation of similar undertakings in the future. The most frequent request was to provide a better location and living conditions, to consult the timing and to reduce the time of waiting, to update the course program, to use audio-visual methods and to make the daily portion of classes shorter. Training participants suggested that shorter, repeatable forms could be organised, e.g. week-end classes. It should be added that companies providing training had to observe external requirements of the bidding procedure. Some of these requirements specified the duration of training or the daily schedule of classes. It is worth stressing that 1/3 of the trainees was entirely satisfied and expressed an opinion that no improvements were needed. This was an element of the high general and specific evaluation of training. Information Technology training The beneficiaries explained the need to attend the courses with such reasons as the necessity to improve their competence, school needs, the necessity to participate in order to achieve other goals, also personal. IT training was delivered by the Consortium of Polish Training Companies consisting of trainers specialised in training teachers and network administrators from four companies: OSI Compu Train S.A, OFEK (Ogólnopolska Fundacja Edukacji Komputerowej) DC Edukacja Centrum Szkoleń i Certyfikacji Komputerowych and Vulcan (the largest Polish developer of educational software). IT training lasted from July to November 2004. The consortium proposed 9 types of courses. The table below specifies the topics and the attendance. 194 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Table 37. Scope and attendance of particular courses Training scope Attendance Training in using the Internet for teaching other subjects – INTN 1306 Two modules of general and basic training in using IT as a teaching aid: PTI1 PTI2 1239 994 Training in basic use of the Internet – INTP 949 Two modules of basic training in IT for school directors – using computers in school and education management: ZAR1 684 ZAR2 616 Two modules of training for network administrators: ADM 1 530 ADM2 482 Total 7716 Source: EPRD report As one can see, most beneficiaries – more than 1300 – attended the training in using the Internet for teaching various subjects and a little fewer – the training in using IT as a teaching aid. The respondents (296 trainees) gave high marks to the range of courses and to their chosen courses as well. More than 21% of the respondents considered the set of courses to be very extensive and nearly 63% as rather extensive. The table below shows how the training adequacy to the needs was evaluated. Table 38. Evaluation of IT training adequacy to beneficiaries’ needs in %* N=296 Positive marks To a rather high degree To a very high degree Negative marks and no mark at all (not at all, very low and rather low, hard to say) Teaching staff in rural areas 60.5 27.7 11.8 Teaching staff in the trainee’s school 61.1 25.7 13.2 The trainee 51.7 33.1 15.2 To what degree does the offer meet the needs of: * Data apply to all IT courses in total, there were no statistically significant differences between evaluation of particular training courses. Source: Consultant’s own analyses A vast majority of trainees stated that the range of IT training was adequate to the needs of the teaching staff in rural areas, the needs of beneficiaries’ schools and the needs of beneficiaries proper. Both the trainers and the contents of training were assessed positively by the trainees. The table and the graph below show the specific outcomes. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 195 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Table 39. Evaluation of trainers * in %, N=296 Rather good Very good Neutral and negative marks (and “hard to say”) Knowledge of the issue 17.0 78.0 5.0 Ability to present information clearly 19.5 71.4 9.9 Commitment to training 14.0 79.1 6.9 Positive marks Evaluation criteria * Data apply to all IT courses in total, there were no statistically significant differences between evaluation of particular training courses. Source: Consultant’s own analyses The data show that a vast majority of beneficiaries were of a good or very good opinion about the competence and the commitment of trainers. Figure 44. Evaluation of IT training based on average marks (on the scale from 1 to 5, where 1 stands for very bad and 5 for very good) 5 4 3 2 Location Training materials Date of training Waiting time Facility of enrollment Knowledge development New skills Usefulness for school work 0 Training duration 1 Source: Consultant’s own analyses High opinions were expressed about training materials and locations where training was organised, as well as the facility to enrol. But the latter, as interviews with representatives of VPIU and gminas showed, was not an easy thing to do. Difficulties followed by protests were caused by the requirement for the candidates to log in via the Internet. One of the respondents from Lubelskie voivodship suggested even that this was a significant reason of withdrawal from participation in any course. This may seem surpising to some degree, as it was possible to enroll by the phone. The trainees stated that participation enabled them to acquire or develop competence in computer operating and in use of operating software and of the Internet. Some of the respondents considered the ability to prepare multimedia presentations to be an important skill. Usually, the new knowledge acquired during the course included creating www sites, using computer for school work management and network administration. Despite the generally positive opinion about the training courses, 22% of the respondents stated that they had not acquired any new competence at all and 11% that they had not developed any of the skills they had already had. These opinions may partly result from 196 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. the fact that 1/5 of the respondents had already attended IT courses. It is possible that some of the respondents had not gained comprehensive information about the contents of particular training and made a choice which was inadequate to their needs. A vast majority of the beneficiaries – 77.7% - use the newly acquired knowledge in their school work. Most frequently – when preparing materials for lessons, including multimedia presentations, when preparing tests to check students’ knowledge, for administration and management activities and a certain group – for network administration. Comments on training organisation Suggestions concerning organisation apply to the fact that the candidates had to wait for a too long time for their courses and that at the same time they were informed of the dates of training too late. Critical comments about the living conditions during courses repeat. As far as the merits are concerned, the most important comment applies to the selection of training groups. The unequal level of competence limited the more advanced persons’ chance to develop their knowledge. Didactic aids and software purchase Not all school directors who were interviewed (although most of them) were interested in applying for this type of support. Some schools do not have the computer equipment needed to use the software that has been purchased. Those gminas that applied for training and for the supplies being discussed here were given preference and had more chance for a higher position in the ranking. As interviews with gminas’ representatives showed, local decision makers were not everywhere willing to allocate large funds from scarce budgets for equipping schools with additional didactic aids. Especially, that some school employees experienced in this type of purchasing stated that the prices of the offered items were not low. To apply to this task required one to understand the importance of the resources referred to for education. Moreover, some directors of schools simply did not believe, that there would be any co-financing for an expenditure like this. This is what one of gminas’ representatives said: “...It has just occurred to me what directors said at the moment when these aids were delivered and they could take them. This was very interesting visually: items for one of the schools were on 20 tables and there were huge parcels there, while some other school was given 2 maps for example. The disproportion was evident. This particularly applied to directors of small schools – I find it hard to say anything, as I do not know the entire project – but they did not know clearly that they would not have to pay for this. Directors, particularly those from small schools, said: ‘I ordered little, because I did not know whether I would not have to pay for this from the school budget”. Most often, school directors explained their decision concerning purchasing with the wish to enhance the quality of education and with the lack of similar didactic materials in their school. It may surprise that 15% of the respondents were not able to justify, why the items had been purchased. Some of them stated that there were more urgent needs in their schools, others – that the aids were seldom used. Primary schools were most willing to purchase materials for teaching foreign languages. More than a half of the schools managed by the respondents purchased one package only; seven packages was the record purchase. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 197 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Didactic aids for teaching mathematics, Polish language and foreign languages were most popular among materials intended for both primary and middle schools. Nearly a half of schools purchased more than one package (from 2 to 5). But some of them purchased even 20-30 packages. Middle schools (gymnasia) were most interested to buy materials intended to supplement teaching natural sciences: biology, chemistry and geography. It is worth stressing that there were some schools that purchased a complete equipment for biological or chemical labs (nearly 12% of our sample of 150 schools). Maps were most popular of all didactic aids – they were purchased by nearly 70% of schools. More than a half of schools bought multimedia programs (available on the list of didactic aids, not on the lists of software). Graphic tables, foliograms, models, musical instruments and optical instruments were also very popular. It is worth stressing that 1/3 of schools purchased sets for correcting faulty posture – medical statistics of such incorrectness are alarming in Poland. The interviewed directors were of the opinion that their schools had lacked the didactic aids that were purchased under the Sub-Component. The decision to buy was most often made by directors (nearly 75%), but in most cases this was on request of teachers. The staff council decided what should be bought in 15% of schools. The fact that there were only few cases when pupils’ parents participated in the decision-making process gives evidence to these schools’ very low degree of co-operation with the local community. The directors evaluated the didactic aids purchased under the Project as rather useful – 38% and very useful – 58%. The remaining few percent expressed negative opinions. The utility evaluation resulted from directors’ conviction that the materials at issue are commonly used in their schools. Nearly 60% of the respondents estimate that teachers use them rather often, and 1/3 – that very often. But 30% of the group of school directors state that they do not use any additional didactic aids in their work. Most often they explain that there are no materials for a particular school subject and that they do not see any need to do this. A vast majority of persons using additional didactic aids – 96% - are able to indicate the resultant benefits. Most often, they mention increased facility and attractiveness of the teaching process, the opportunity to expand students’ knowledge and to enhance their activeness. Apart from didactic aids, schools could also purchase instructional software. Almost all schools managed by directors participating in the survey have their separate computer rooms – 87%, or computers are available in classrooms – 12%. The average rate of students per computer is 23, the larger the school the worse the proportion. Students may use computer workstations after classes in 90% of schools. Almost all schools are connected to the Internet, but 42% of them do not have a fixed connection and the facility of using the Internet is limited there. Before being offered the chance to apply for co-financing under RDP, 65% of schools had didactic software. Most often, this included encyclopaedias and atlases (48%), instructional packages for particular subject (27%) and dictionaries and programs for teaching foreign languages (24%). 198 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. In case of primary schools, software for teaching Polish language and mathematics were purchased in the first place and for integrated teaching less often. Usually, schools purchased one or a few packages, although some of them ordered 40 or even 50. As far as the software intended for primary schools and middle schools (gymnasium) is concerned, packages for teaching artistic subjects, foreign languages, natural sciences (integrated packages), book reading education and media-related education were purchased most often. Usually, schools did not buy many packages, but in single cases even 30 packages were ordered. Middle schools (gymnasia) – similarly as in case of didactic aids –usually choose packages supporting teaching of natural sciences, geography and mathematics. School directors substantiated their choice of particular software packages with the need to enrich teaching various subjects and to make it more attractive, the needs reported by teachers, the need to supplement the software they had already had and with the availability and financial affordability of the software offered under the Project. Most often, the software was chosen by school directors and teachers who were supposed to be its main users; the decision was made by the teaching staff council in fewer cases (14%). The respondents gave high marks to the quality of the purchased software; it was graded as rather useful in 55% and as very useful in 39%. Only 6% of the respondents were of an opposite opinion. In the directors’ opinion, the software is used rather often – 65% and very often – 10%. In the respondents’ opinion, the use of didactic software is mostly limited by the shortage of suitable computers (31%), by the lack of computer-related competence and by teachers’ concerns (23%), as well as by the little usefulness of the packages (11%). Nevertheless, 72% of directors declare that they use the didactic software themselves. The respondents referred to such benefits of using didactic software as students’ increased activeness, the possibility to make the process of teaching easier for the teacher and to expand students’ knowledge. Comments on organisation of the purchasing procedure Most comments were about the tasks’ too long duration – representatives of gminas criticised the idea to organise the central bidding in particular. Some comments applied to excessive costs – the respondents stated that in case of such a large transaction prices should be lower than wholesale prices known to them. Regional consultants referred to complaints of special schools directors’ who did not find any aids for disabled students on the list of materials being offered. This should be considered as a significant oversight on the part of MONES, as this ministry had approved the list of materials. Use of Scholaris Internet portal Presently, 11% of teachers who attended didactic training and who filled the questionnaires, use the portal. Most often, they look for information concerning their discipline and use of didactic aids, for scenarios of classes and for any interesting information or pedagogic news. It may seem surprising that so few school directors use the educational portal – less than 18%. The proportion of portal users is highest in Lubelskie voivodship – 35%. School Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 199 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. directors look for current information and materials for lessons, often – for lesson outlays. Some directors and teachers declare that they read publications presented in Scholaris. 14% state that they only browse the site. 3.3 Summary 3.3.1 Cost-Benefit Analysis Evaluation of efficiency According to what gminas’ representatives stated, all beneficiaries had been trying to minimise their own costs, apart from the financial participation envisaged in the Project. This is why the Project was implemented by employees as a part of their regular duties at the level of gmina. The respondents expressed an opinion that such administration costs as business travels, phone calls, paper, etc. were not high. A vast majority of gminas decided to cover their own contribution from the funds they had at their disposal at the moment. Only few respondents referred to loans offered by the voivodship. Similarly, there were few examples of attempts to raise funds from various local institutions or even from inhabitants. It is difficult to estimate, how much additional time the beneficiaries had to spend on work relating to RDP. School directors participated in filling applications and supervised refurbishing works. Even those persons who considered the co-financing to be too low, agreed that without this support they would not be able to effect the investments referred to. Building companies were selected for refurbishing assignments on the lowest price basis. According to a vast majority of gminas’ representatives and school directors, the quality of workmanship raised no objections. It will be possible to quantify economic benefits resulting from such improvements as thermal insulation of buildings after the first heating season. Costs of training should be regarded as moderate. The comparison with other companies, both private and state-owned teachers development centres – shows the advantage of STO, considering particularly the quality of training which was confirmed by trainees’ assessment. But, although the cost that some teachers had to incur individually was relatively low (around PLN 200), it was twice as high as the price paid by students of post-graduate programmes financed by the Ministry of Education and Sports in the form of a grant. Moreover, the trainees paid for travelling to the training location on their own. Some of the respondents (representatives of gminas, school directors) found the prices of didactic aids and software relatively high. It is not a simple thing to compare the prices here, but the co-operation was established with renowned companies who offered a range of products adequate to the Project needs. The choice of such companies was limited on the Polish market and one should state that it was correct. It is not easy to draw up a balance of Sub-Component B2 costs and benefits in case of training effects or development of the didactic base. It depends on the beneficiaries to a high degree. It is difficult to estimate, to what degree they will use the new potential. One should expect that the quality of teaching will be higher, but it is difficult to predict the 200 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. scale of this change responsibly. In case of training, the cascade effect (knowledge sharing) intensifies the benefits. Similarly, it is difficult to compare the costs of starting Scholaris portal with the intangible benefits in the form of opening the educational forum which is also used for exchange and dissemination of professional experience and opinions. 3.3.2 Long-Term Impact of Sub-Component B2 Evaluation of Project effects sustainability Tasks of Sub-Component B2 was expected to result in two types of effects. The first type includes immediate effects, the second one – a long-term impact. The material support provided to beneficiaries enabled them to effect rehabilitation works within the scope described in the present document, to purchase didactic aids and/or software. These are physical, observable changes that would not occur – not to such an extent, at least – without RDP. The next group of effects, easy to predict, includes economies resulting from thermal insulation of buildings where windows were replaced, roofs were mended, etc. Effects of this type can not be evaluated in quantitative categories only. Physical conditions in which pupils learn are very significant for the quality of teaching. The present sub-section of the study will present qualitative long-term impact first of all. These can be found in the rehabilitated school base as well. Refurbished and equipped common rooms should serve students for many years. For many of them, this may be the best place to play safely, to make their homework under care of helpful teachers, to spend their free time in a manner conducive to development. Refurbishing and modernisation of school facilities included also building gymnasia (sports halls) that are still lacking in many small schools. There is no need to substantiate the importance of supporting children’s physical development or benefits resulting from organisation of their free time by the means of physical activities. Since September 2004 six years old children have been subject to compulsory education. Usually, education reformers do not leave much time to gminas managing the local education to prepare for new tasks. Hence, the idea to include this new task should be appreciated, although there was less interest in it than in case of previous tasks. But well equipped pre-school education classrooms for youngest inhabitants of small farming gminas should provide a better educational start to children in rural areas, where very few children go the kindergartens. Training and supply of didactic materials and educational software had been designed in hope that they would make it possible to achieve specific qualitative indicators in the field of improving the quality of education, such as: • better, team work-based school management, • more interactive elements in teachers’ work, • widened scope of teaching methods, including activating methods, • more diversified approach to teaching, through wide use of didactic aids, • use of information and computer technologies in teaching, Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 201 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. • increased interest in purchasing up-to-date teaching tools using multimedia technologies. Scholaris Internet portal was expected to strengthen the chance for achieving all these indicators through interactive participation of education personnel and clients in its creation and use of education-related information and products presented there. The refurbishing and modernisation program, besides the improvement of school facilities proper, was expected to contribute to the increase of education resources utilisation in rural gminas through so-called rationalisation of the teachers/students rate (i.e. increasing the number of students per one teacher). Most participants of training courses for school directors assure that they have acquired and use in practice the skill to organise a group for active team work. Similarly, teachers who attended didactic courses declare, that they use the knowledge acquired during the courses in their day-to-day school work, particularly – activating methods, new methods of grading and wider use of didactic aids and new solutions of educational problems. Most teachers shared their new knowledge with their colleagues and some of them initiated teams developing new curricula in schools, educational activities or after-classes activities. What is also important, the conviction has been deepened that life-long education of teachers is a necessity. Participants of IT training are willing to use computer technologies in teaching and in their administration work more often. Shortage of equipment and suitable premises may be a barrier. Further development of didactic infrastructure will depend on the financial condition of rural education to a high degree. And this is most determined by subsidies for education and by the growth of gminas’ wealth which is in conjunction with the condition of economy on the macro scale. Nevertheless, teachers’ belief in the value of using various didactic aids in their day-to-day teaching work and their pressure to gain these aids may enhance the motivation of local decision makers to purchase what is needed. The sustainability of investment effects should be estimated at a few years or at very long years, depending on the scope of work. The sustainability of the effects of training and purchasing should be strengthened by the means of further activities, continuous refreshing of the didactic base and continuous development of teachers’ competence. One should also remember about factors that reduce the sustainability of these effects, such as: changes in curricula (currently, MONES is working on changes in the teaching programme basis) and changes in the teaching staff employment (it is planned that a large group of teachers will retire in 2006). 3.3.3 Activities of Sub-Component B2 Sub-Component B2 was implemented in two phases. Phase I: rehabilitation and investment tasks including: 202 • refurbishing, renovation and modernisation of primary schools, middle schools and special schools buildings; • renovation and modernisation of school common rooms. • renovation and equipping pre-school education classrooms (II stage of phase I – since May 2003) Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Phase II: training and didactic tasks including: • didactic training for teachers and school directors; • IT training; • purchase of computer software; • purchase of education materials and equipment (teaching aids). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 203 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Figure 45. Schedule of Sub-Component B2 Source: Consultant’s own analyses based on Interim Reports of Project Management, PCU 204 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. Implementation of Sub-Component B2 began in 2001 and most of the tasks were completed by October 2004. At the beginning of the year 2001 rules of Sub-Component implementation, criteria of evaluating applications for co-financing of refurbishing tasks and deadlines for applying were specified. Ranking lists were approved at the turn of May and June 2001. The value of applications exceeded the co-financing potential 6 times. First of all, applicants requested for co-financing of refurbishing and modernisation of school facilities; moreover, the application procedure for school common rooms had to be repeated. First contracts for co-financing were signed on 31 May 2001 in Świętokrzyskie voivodship and first contracts with service providers were signed in July. First works were completed by the beginning of the school year 2001/2002. As some economies resulted from bidding procedures, projects from more distant ranking positions were accepted. Cofinancing contracts and service contracts were signed all over 2002. At the same time, relatively many problems were encountered during the bidding procedure for “Minor Works” (Małe Roboty). These caused a delay in procedures. PCU submitted the first application for disbursing funds to gminas in September 2001. In July 2001 a package of documents was prepared for the purpose of applying for didactic and IT training and for purchasing educational materials and software. The application procedure, preceded by training how to apply, was conducted among school directors, teachers, representatives of gminas and powiats at the beginning of 2002. In June, the applications were submitted to the Regional Steering Committee (RSC) to draw up ranking lists. It tuned out that there was less interest in training than in purchasing educational materials and software, therefore MONES applied to the World Bank for reallocation of funds. A positive response to the application was given in September 2002. In January 2002 EPRD company was selected to provide technical assistance in Project implementation and monitoring. With regard to delays in Sub-Component implementation, contract execution was suspended until September 2004. EPRD organised a team of Regional Consultants whose task was to perform implementation activities and training actions for the personnel of gminas. The team provided consultations in preparing and submitting applications and technical assistance in analysis of applications to VPIU. Moreover, the team participated also in preparation of bidding documentation for training. Besides, EPRD carried out a qualitative evaluation of the Program outcomes. In October 2002 a bidding for creation of the Internet Centre of Education Resources was settled. The project was assigned to OSI Compu Train S.A. The bidding for computer equipment delivery was settled in November 2003. In May 2003, the National Steering Committee (NSC) adopted a resolution to allocate additional funds for renovation and equipment of schools and school common rooms from the Project reserve (EUR 7 million) in 9 new voivodships. The work to develop application evaluation criteria and to construct an appropriate form began immediately. At the end of 2003 applications were collected and ranking lists were drawn up at the beginning of 2004. The investment works were continued even in the third quarter of 2004. The bidding for delivery of software for schools was settled in December 2003 and in the same month Young Digital Poland S.A. commenced delivering the software. In the second quarter 2004, voivodships signed contracts with the provider of didactic training – Społeczne Towarzystwo Oświatowe (STO). The recruitment of trainees and the pilot training was conducted completed by the end of June. In the third quarter of 2004 contracts were signed with the Consortium of Polish Training Companies – the IT training Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 205 Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. provider. The Consortium commenced implementation of IT training in July 2004. STO completed its training in September 2004 and IT courses were continued even in the fourth quarter of 2004. At the end of the third quarter of 2004 contracts for delivery of education materials and equipment were signed and deliveries were completed by the end of October 2004. When assessing the Sub-Component activities one should stress the efforts leading to reallocation of funds from tasks that were less desired by the beneficiaries to those for which a higher demand was declared. The procedures that protracted the time of waiting for training and for purchase of materials and software should be assessed definitely negatively. This is particularly important, as according to the organisers’ interesting conception, participation in training was supposed to be interrelated with purchasing and to lead to the global effect of improved quality of teaching in rural primary and middle schools and better use of school resources, these effects leading to achievement of the Sub-Component’s basic objective. As far as the synergy of objectives and activities of the Sub-Component and the whole project, it is worth mentioning two detailed and one basic issue. The first specific issue was the positive preparatory training (administration of the application procedure) for gminas participating in Component C. The second, indicated by representatives of gminas, was the need of better synchronisation of activities relating to the school network consolidation and to the process of building roads for transporting children to schools. The basic issue was the matter of increasing chances for human capital development through coherence of activities designed for all Sub-Components. Building the necessary infrastructure and supporting small business means building the labour market for future graduates of rural schools. Education quality improvement and better education of individuals translates into reduction of unemployment. Developing competence of people leading local self-governments means more efficient functioning of all regional institutions and experience gained in the Project should be used when applying for EU funds. Data obtained from interviews with representatives of VPIU and gminas show that it was difficult to achieve such a serious effect of synergy mainly due to financial limitations of local self-governments and due to organisational reasons. 3.3.4 Conclusions and Recommendations for the Future All respondents from gminas participating in RDP stated that RDP was an exceptional project in terms of sensitivity to the education-related needs, simplicity of applying and professional preparation and support provided to beneficiaries. Although in comparison with other programs, the proportion of co-financing was rather high, the beneficiaries wish that the financial conditions were even better and the global amount of co-financing higher. Education needs are immense in rural areas and it is hard to satisfy them without a giant financial support. RDP objectives and activities were adequate to the needs and these good ideas should be continued. RDP beneficiaries suggested that the number of beneficiaries could be increased in future similar projects, if possible. Small schools that can not be closed due to geographical and social reasons are also in need. Representatives of gminas think that kindergartens and secondary schools need support as well. These judgements are justified, as the number of kindergartens and children going there in rural areas in Poland is one of the lowest in Europe and this fact is a certain reason of later disparities in the 206 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B2 Anna Wiłkomirska, Ph. D. field of education. Secondary schools, particularly those with vocational profiles, require serious restructuring, but there are no funds for this. Representatives of VPIU proposed that bidding procedures could be held at the voivodship level and some of them suggested even that the this authority should be delegated directly to gminas. This would reduce the time of waiting for results of the bidding and would eliminate all negative consequences of delays that occurred in the Project this time. When thinking about further support to education in rural areas, it is worth considering the following activities: • Further, continuous modernisation of the school infrastructure – buildings, facilities and the didactic base as well. • Organising various forms of teachers’ continuous training, not only by the means of external courses, but also through reconstruction of regional (voivodship) and district (powiat) training centres that might offer training teachers directly at schools and organise local conferences enabling exchange of experience. • Encouraging Polish universities to co-operate in the field of teachers development and also to develop an educational offer intended for students of middle and secondary schools – future university students; to organise voluntary activities supporting education in rural areas (e.g. actions of the “student with a class” type). • Offering a wide and inexpensive range of educational products, activating students’ extracurricular cognition development, building their confidence in educational success, facilitating them access and stimulating them to self-development by the means of extracurricular activities, in the library, via the Internet. • Providing a wide access to the Internet to schools (fixed connections) and the possibility of regular refreshing and upgrading computer hardware and software. • Providing support to small schools, so that their isolation is not deepened and the appropriate education level is ensured. • Grants and scholarship programs managed locally – schools might construct education quality enhancement programs and raise funds for them; grant programs should be extended, but made dependent on students’ achievements, not only on their material situation. • Continuous monitoring of students’ school achievements, analysis of outcomes of external final exams held in middle and secondary schools and analysing students’ planned education careers; motivating them to extend their education paths. • Voivodships’ assistance in encouraging local self-governments (powiats and gminas) to establish contacts with schools from the nearby large cities and with similar rural schools abroad, on a similar basis as city schools do. Without the continuous stimulation and financial support to the above activities aimed at education in rural areas, the quality of school work – and consequently, the quality of graduates – will not be enhanced so that in the future they may become a large group ensuring improvement of economic and social conditions of living and working in regions that are subject to marginalization today. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 207 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka 4 Sub-Component B3 Beata Ciężka Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 209 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka 4.1 Top-down Analysis 4.1.1 Identification of Problems Sub-Component B3 Responded to The analysis of Project documents shows that (despite the fact that there is no such provision in the Project design documentation), the low development level of rural areas and their inability to function in the European Union structures were the basic problems RDP was intended to address. The objective of Sub-Component B3 was to increase the level of efficiency and effectiveness of local and regional administration (gminas, powiats and voivodships) within the voivodships covered by RDP through institutional development, training programs and capacity building measures and thereby improve the conditions for creation non-farm employment in rural areas. In particular, the following problems were identified with regard to the self-government administration in PAD: • lack of expertise in local and regional administrations in activities supporting local and regional development, including development of specific development plans for their gminas, powiats and voivodships in particular, • limited involvement of citizens in the process of governing caused by so-called culture of passivity and the existence of few formalised opportunities to participate in the process of making decisions that influence the life of all citizens, • authorities representatives’ and self-government professionalism and inability to use the best practice administration’s lack of In case of the problem defined as the lack of expertise in local and regional administrations in activities supporting local and regional development, one should stress that this concerned mainly new powiats and voivodship self-governments established as a result of the administration reform. This aspect was addressed when defining the source of problems as “decentralizations of public services and finances to two levels of democratically elected self-government authorities: powiats and voivodships”. It was also stated that “decentralisation of public services and finance to local self-governments is placing great demands on public administration, especially in rural areas”, as “councillors and board members in rural areas often do not have the necessary organisational and managerial skills, while there are new functions to administer and new processes to apply”88. The next problem was diagnosed as limited involvement of citizens in the process of governing, described as the culture of passivity. This means that in RDP authors’ opinion, the not entirely formed civic society, i.e. the fact that there is no real community of inhabitants, aware of their responsibilities and working for the local community effectively, is a barrier which makes it impossible to build a model public administration institution. The culture of passivity is deepened by the scarcity of formalised opportunities to participate in decisions which affect citizens’ lives89. 88 POLAND – POLAND RURAL AREAS DEVELOPMENT PROJECT; Project Appraisal Document, p. 3. 89 PAD, p. 32. 210 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka On the other hand, the problem of insufficient professionalism in the Polish public administration was indicated in a manner referring a wider organisational context, including the Constitution, stating that the administration is too bureaucratised, far from the managerial model and too limited by the brace of regulations and the traditional system of power, where decisions were made based on political arguments, instead of economic reasons. The tendencies observed in the Polish self-government administration departed considerably from changes that occurred in most European administrations and all over the world in the nineties. It also turned out that the Polish administration had a significant problem of the too weak institutional potential of self-government administration manifesting itself in the lack of appropriate organisational structures, operating procedures, mechanisms ensuring the local community participation in 90 managing public matters and the lack of competent, efficient officers . Apart from these problems, such issues as the shortage of knowledge about development of teleinformation technologies and the inability to use this knowledge in administration’s 91 activities were significant as well . Polish administration’s limited access to latest information technologies was the basic barrier to developing a “modern institution”. This reduced significantly the ability to process information quickly and effectively. Another problem PAD referred to was administration’s unadjustedness to play new roles in the changing economic situation. In its new role, the administration is expected to have a serious influence on achieving a significant pace of civilisation development. To play this role, it has to strengthen the quality of its work and to make the work of all public services and forces in its territory more efficient. The situation assessment for the purpose of Sub-Component B3 did not include any definition of problems that restrain creation of new jobs in non-farming professions in rural areas. It is also difficult to find any direct references to the role that should be played by self-government in creation of non-farm employment in rural areas. Moreover, no diagnosis was made to indicate the direct relation between the efficiently working selfgovernment administration and the economic development, with a particular focus on creating new jobs in the off-farming sector. The definition of these problems may be reconstructed indirectly through analysis of tasks planned for Sub-Component B3. The description of tasks includes an assumption that institutional development of public administration will contribute to activation of the private sector. So, the conclusion was that the outdated public administration – both central and self-government as well – that had not been modernised, was a barrier to development of non-farm employment. One may conclude from this reasoning that the opportunities for the private sector development grow along with the growth of public administration’s support to the market economy, creating sound policies and providing public goods and services in a costeffective manner. Evaluation of Sub-Component rationale The rationale of Sub-Component B3 has been evaluated on the basis of analysis how problems were defined and what role was given to Sub-Component B3 in the implementation of RDP’s other Components. The diagnosis of the key problems has been assessed with reference to the National Strategy for Rural Development – the basic strategic document specifying goals of Poland’s regional development for the years 2001 – 2006, i.e. the period when RDP was implemented. The analysis refers also to other 90 Report of the Institute for Market Economy Studies, p.10. Zarządzanie wiedzą w społeczeństwie uczącym się; Centrum Badań nad Edukacja i innowacją; OECD 2000. 91 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 211 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka studies where regional development problems were diagnosed with a special focus on rural areas development. In case of the problem defined as lack of expertise in local and regional administrations in activities supporting local and regional development, the authors of PAD defined the problem correctly. It should be stressed here, that voivodship selfgovernments play the key role in regional development. This was their responsibility to work out development plans at the moment when the program of support to be provided under RDP was being created, to provide a basis for the contracts to be signed by Marszaleks with the Government. It was much more difficult for powiat self-governments to find their place, particularly in the area of relationship between powiat self-government and gminas – especially rural gminas and tasks to be performed for them. The indication that powiats should be created was in the constitutional rule of auxiliarity. According to this rule, powiats were supposed to perform such tasks for gminas that these were unable to perform adequately due to organisational and economic reasons. Delegation of tasks to powiats was expected to increase the efficiency and the effectiveness of public administration institutions’ activities and to improve the quality of services provided by them. It should be noted here that in terms of RDP implementation, the main task of powiats was to stimulate occupational activeness of the rural population. Powiat boards (Starostwo), acting through Powiat Labour Offices, were supposed to analyse the local labour market and to implement necessary programs resulting in counteracting the unemployment, including long-term unemployment, and to activate rural inhabitants in undertaking non-farm jobs. Should the effectiveness of Powiat Labour Offices’ work be measured by the rate of unemployment level in rural areas, the effects of their activities were insignificant, despite the support provided under PHARE 2000, 2001, 2002 Human Resources Development Program in the years 2002, 2003 and 2004. The situation was similar in case of other tasks performed by institutions reporting to Powiat Boards, such as Powiat Centres for Family Assistance or secondary schools. It is worth adding that the Union of Rural Gminas of the Polish Republic representing selfgovernments from rural areas, expressed its dissatisfaction with the performance of tasks by powiat administration and this was manifested in a resolution authorising the Union’s board to take steps aimed at liquidation of powiats. In case of the self-government administration of the basic level (gminas), the problem of lack of expertise in local and regional administrations in activities supporting local and regional development was defined as the lack of public administration representatives’ competence – particularly in rural areas – in working out and implementing development programs. This problem was identified in PAD and in the National Strategy for Regional Development as well. The Strategy states that selfgovernment administration has had no sufficient potential to meet the new challenges imposed by programs relating to Poland’s accession to the European Union, indicating that "it seems in this situation that the significant dispersion of funds available to territorial self-government among more than 2.5 thousand subjects and the technical potential to prepare a sufficient quantity and quality of projects aspiring for co-financing from assistance funds is the more realistic barrier to absorption of assistance funds”. It is difficult to state now, how significant the problem was. Should one assess the preparedness of self-governments, including rural ones, on the basis of applications submitted by them to obtain co-financing under various projects in the period 2000 – 2004, one may say that this was not a problem for them. When analysing the effects of SAPARD program only, one may say that gminas submitted more applications for 212 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka investment tasks that the program funds had envisaged92. The implementation of IOPRD proved that self-government administration, including rural gminas, did not have any problems with preparing applications, despite their increased complexity and new requirements relating to the need to prepare a feasibility study. Interviews conducted with representatives of self-government administration show that a decision to assign funds depends not on the quality of applications, but on voivodship’s financial potential first of all. The territorial self-government units’ capability of financing investment expenses was the problem which the authors of PAD did not take into consideration and which was stressed in the National Strategy for Regional Development. In the opinion of Strategy authors, this should not be a barrier to absorption of EU assistance funds globally. According to data of the Institute for the Market Economy Studies, in the years 2001-2006, the territorial self-government will be able to spend some PLN 69.5 billion on new investment tasks financed from its own funds. One-third of these funds (around PLN 23 billion, i.e. EUR 5.75 billion) could be allocated for co-financing projects co-financed by foreign assistance funds and the Budget. This amount is sufficient to provide co-financing of regional development at the level of 25%-50% in the whole period covered by the Strategy. One can conclude, how real the problem was, on the basis of effects achieved as a result of such programs as PAOW, PHARE 2000, PHARE 2001 and SAPARD. Territorial selfgovernment units, including rural gminas, did not have any serious problems with performing the investment tasks. It should be noted that despite the fact that in the years 1998 – 2001 a decrease of investment expenses took place in small and medium-sized gminas as a result of a fall in self-governments’ incomes, a significant revival of 93 investment activities was observed in rural areas in the years 2001 – 2003 . The geographical pattern of the territorial self-government units’ financial potential is the next problem related to Poland’s accession to the European Union which has been identified in the National Strategy for Regional Development. The document says: “the largest ability of self-governments to finance development activities counted per inhabitant can be found in Mazowieckie voivodships, while the smallest - in WarmińskoMazurskie. Moreover, significant disparities of projects co-financing abilities exists within individual voivodships too, e.g. the financial potential of gminas in the district of Warsaw 94 is incomparably higher than in case of small gminas in Mazowieckie voivodship . In 1999, the highest productivity rate was noted in: Mazowieckie, Śląskie, Pomorskie, Dolnośląskie and Wielkopolskie voivodships, while it was lowest in Świętokrzyskie, Lubelskie, Podkarpackie and Podlaskie. The level of competitiveness and development was highest in Pomorskie, Wielkopolskie, Mazowieckie, Małopolskie and Śląskie. They owed their position to the high level and efficiency of the production sector, to human resources potential and to the relatively well-developed infrastructure. Świętokrzyskie, Warmińsko-Mazurskie, Podlaskie, Lubelskie and Podkarpackie were evidently weakest 95 regions in terms of efficiency, infrastructure development and living standards . When compared with the EU regions, Świętokrzyskie, Lubuskie, Podlaskie, Podkarpackie and 92 Poland MID-TERM Evaluation of the SAPARD programme in Poland for the implementation period 2000-2003 (PL-7-05/00) Ref.: EUROPEAID/114803/D/SV/PL Final Report December 2003 AGROTEC SpA. 93 Paweł Swianiewicz, Finanse lokalne - teoria i praktyka, Warszawa 2004, p. 205. 94 NSRD, p. 79. 95 NSRD, p. 13. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 213 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Warmińsko-Mazurskie were economically weakest in 1999 (their GNP ranged from 24% 96 to 26% of the average EU level) . Citizens’ limited involvement in the process of governing described as the culture of passivity was the next problem the authors of PAD referred to. It should be stressed that the problem of population’s participation in decisions of local authorities should be analysed with the size and the type of gmina taken into account. According to studies, inhabitants of rural areas placed much more confidence in self-government authorities 97 than inhabitants of cities did . Besides, they were more deeply convinced that these authorities act in the interest of the local community and not in their own interest. For example, rural dwellers think that the problem of corruption exists at the central 98 administration level first of all – not on the self-government level . The self-government election attendance rate which is higher in the country than in cities gives evidence to the rural population’s greater involvement in the local community life. Inhabitants of rural areas concentrate on solving problems of the local community rather than on participation in “great politics”, therefore the attendance rate of the parliament election, presidential election or the latest European referendum was lower in rural areas. Rural inhabitants’ participation in civic activities is different in nature than in case of inhabitants of cities. Most activities are inspired by self-government authorities in rural areas and to a much lower degree by non-government organisations. The role of NGOs in the local community functioning is different in rural gminas than in cities. The research made by Jan Herbat shows clearly that associations and foundations are typical NGOs in cities, while in case of rural areas, all types of “action” organisations are more active – voluntary fire brigades, village women’s clubs, all types of committees set up to build a 99 water supply network for example . The research shows that inhabitants of rural areas 100 are more involved in social activism than inhabitants of cities and that they are more willing to work for the community, to co-operate with other people and to participate in 101 political activities . On the basis of these facts one may advance a thesis that an accelerated process of civic society building occurs in rural areas in the classical meaning of the word, although of a different nature than is observed in urban environments. The lack of the professionalism in the Polish public administration was another problem indicated by the authors of PAD. But studies on the self-government administration functioning showed a clear difference between rural areas and cities. Rural dwellers were more satisfied with the manner how self-government officers treated them. On the other hand – according to PAD – self-government administration in rural areas had problems 102 with drawing up investment plans . Some gminas planned their investments in a cycle which was not longer than one year. The fact that rural self-governments were not using a cost-benefit analysis, the IRR and the NPV when planning their investments was 96 NSRD, p. 9. Paweł Swianiewicz, Życzliwa obojętność. Władze lokalne i reforma samorządowa w oczach opinii publicznej w krajach Europy Środkowej; Warszawa 2001. 98 Op. cit., p. 61. 99 Jan Herbat, Empiryczne modele społeczeństwa obywatelskiego w Polsce. (Empiric Models of Civic Society in Poland” ; master degree thesis; Institute of Socilogy of Warsaw University. Warsaw 2004. 100 Polska wieś 2002. Raport o stanie wsi. (“Polish Rural Areas 2002. Report on the Rural Areas Condition”) Fundacja na rzecz rozwoju polskiego rolnictwa (Foundation for the Polish Agriculture Development), Warszw 2003, p. 75. 101 Polska wieś 2002, op. cit., p. 76. 102 Polska wieś 2002, op. cit., p. 69. 97 214 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka another problem identified correctly in PAD. It was a necessity to use this method when applying for EU or World Bank funds. Sub-Component B3 was expected to play a special role in terms of RDP implementation cohesion. Apart from its typical tasks, i.e. implementation of the Institutional Development Program, the Training Program and the support to the Ministry of Interior and Public Administration (MIAPA), tasks of a “horizontal” nature were designed for the SubComponent. It was assumed that a special stress will be put on public management and administration services during training courses and this was supposed to have a significant impact on the successful implementation of RDP and its program Components. Horizontal activities of Sub-Component B3 were planned to include surveys intended as a link between activities in support of self-governments and other Components of RDP. This Sub-Component activities were designed to co-ordinate activities with other program Components and to ensure quality control, including mechanisms of monitoring and assessment. Comparison of Sub-Component B3 with other Similar Programs Sub-Component B3 was the first co-ordinated and large government program improving management in self-government administration and addressed to gmina and powiat administrations directly. Earlier, only international bilateral projects had dealt with these problems: USAID “Local Government Partnership Program”, British Know How Fund “Support to Self-Government” and indirectly – few activities performed under Phare, addressed to small groups of officers and implemented in selected local selfgovernments. The philosophy behind these programs was to teach the selected selfgovernments the best management practice. In the future, these units were expected to become a germ of the best practice dissemination in neighbouring units of territorial selfgovernment. The experience gained as a result of nearly 15 years of these programs functioning shows that the effort to disseminate these locally developed best practices on a larger scale was unsuccessful. Usually, assistance was provided to the same selfgovernments under various programs, as they developed a skill to obtain support from foreign assistance programs throughout these years. At the same time, organisations representing local self-governments were not able to work out a uniform training program for their members. All training programs that were provided were of an immediate, short-term nature. They were designed to provide knowledge necessary to implement acts and laws. At the same time, these organisations Have not developed any consultancy system for gminas, similar to those existing in many EU countries. On the other hand, what seems obvious in the Polish self-government tradition, the Government has not developed any well thought-out policy of systematic training and consultancy for gmina and powiat self-governments. On the other hand, selfgovernment activists would regard a policy like this as Government’s interference in their internal affairs. “Self-governance” was the basic value for self-government administration, activists and authorities in the last 15 years. Self-governance identified as a strict separation of territorial self-government units functions from the self-government administration. The opinion was that the rules of government administration functioning did not apply to the self-government administration. This is how one may explain selfgovernment activists’ reluctance to introduce a competition procedure of appointing officers similar to that which is applied in the government administration. It seems that these views were confirmed by politicians who assumed that the self-government administration was managed optimally in Poland and if it was not, than this was selfgovernments’ problem. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 215 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Sub-Component B3 was the first program which, despite the lack of adequate legal regulations, was intended to introduce procedures existing in the government administration – the public administration officer’s code of ethics, competitions for posts in the public administration, the social dialogue rules and modern management methods. It seems that the direction of proposed activities was correct, as this issue has recently become a subject of Parliament’s work. When designing assumptions of Sub-Component B3, its authors considered that appropriate absorption of co-financing from pre-accession and structural funds would be one of the problems. Thus, the Government manifested the concern that the insufficient preparedness of self-governments – the largest future beneficiary of structural funds – would make a net payer of Poland. A very precise outline of the training program results was very important in terms of the participants commitment to the Project – it was required to prepare projects that would be submitted in the form of an application for pre-accession or structural funds thereafter. This aspect was absent from other training programs relating to implementation of modern management solutions. Many training courses organised by international organisations under bilateral projects or by Phare had been intended to make the territorial selfgovernment units familiar with latest solutions in the area of management, but using them in practice had not been their task. Therefore, it is hard to appreciate the fact that this was the first program intended to improve the management practice, which had its practical dimension. Similarities to other programs emerged in the approach to dissemination of outcomes. Similarly as in USAID “Local Government Partnership Program”, partners were selected among the national organisations representing territorial self-government units. The assumption was that there is a convergence between the mission of Sub-Component B3 (building self-governments’ institutional potential) and the mission of these organisations and their obligations to their members. The intention was to use the partnership of the Government and self-government parties in the Joint Commission in practice. The Project authors were of an opinion that this was the right platform for creating legislative frames for implementation of modern management methods promoted in the form of training and consultancy by Sub-Component B3. These organisations were entrusted with the entire program of Sub-Component B3 achievements dissemination. The organisations were responsible for organising regional conferences, conducting a competition titled “Local Self-Government Management Leader”, launching the “Good Management Practice Base” in the Internet and up-dating it. As far as the Dissemination Program is concerned, MIAPA participated in regional conferences and promoted materials worked under SubComponent B3 on the ministerial web site. Representatives of MIAPA participated also in the “Self-Government Management Leader” competition commission and in the group of experts. The change in the approach to implementation of Sub-Component B3 was expected to manifest itself in the role to be played by MIAPA. Implementing RDP, MIAPA was supposed to become a member of a group of those governmental institutions in EU and OECD countries, who make an active contribution to improvement of services provision quality and efficiency. The belief that the quality of service provided by self-governments becomes an element of the entire system of tasks performed by the public administration and that it determines all citizens’ attitude to the State was right. This was expected to change the existing situation, and changes were to be initiated by a government institution instead of foreign governments that had been providing assistance to Polish self-government. There is one more noticeable difference in the approach to implementation of the program of changes, which is represented by bilateral assistance programs and Sub216 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Component B3. Bilateral programs had been promoting solutions from their own countries first of all. Some of them promoted under USAID program were far from solutions applied in EU countries and could not be applied in the practice of Polish selfgovernments and with their legal status. The activities performed under Sub-Component B3 complied with the “new public administration” paradigm. The program adopted models from many countries to the conditions in which Polish local self-governments worked, keeping to the basic rules and standards of the “new public management” at the same time. These rules and standards are as follows: • Making managerial solutions similar to the practice of market organisations; • Introducing effectiveness and efficiency assessment measures; • Strengthening administration’s servicing function (the function ancillary to citizens); • Using new methods that enhance public participation.103 Another difference in approach between Sub-Component B3 and other programs of this type can be found in the expected outcomes. It was difficult to define the impact on the Polish self-governments’ functioning which had been expected in case of USAiD and Know How Fund, while the implementation of Sub-Component B3 was supposed to result in increased effectiveness in public expenses control and in high quality of services, on the condition that adequate legislation framework would be created. The currently effective act on TSU financing was the first step in this direction. 4.1.2 Evaluation of Sub-Component B3 Objectives and Activities Objectives of Sub-Component B3 referred to the general objectives of RDP that were defined as: (i) increase the level of off-farm employment in rural areas, (ii) contribute to the on-going decentralisation of self-government and regional development and (iii) assist Poland in building institutional capacity to absorb EU pre-accession and structural funds. Thus, the overall objective of Sub-Component B3 was to increase the level of efficiency and effectiveness in eligible units of local and regional administration (gminas, powiats, voivodships) within the voivodships covered by the RDP, through institutional development, training program and capacity building measures, and thereby improve the conditions for the creation of non-farm employment in rural areas. Achievement of this objective was expected to improve the provision of goods and 104 services to the private and the public sectors, thus facilitating economic activities. . 103 Diagnoza stanu terytorialnej administracji publicznej w Polsce (“Diagnosis of the Territorial Public Administration in Poland”), editor: J. Szlachta, J. Zalewski, Institute for Market Economy Studies, 2002, Warszawa, prepared under RDP – Sub-Component B3. 104 PAD, p. 68. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 217 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Sub-component activities Sub-component Project Objectives objectives Figure 46. Sub-Component B3 objectives and activities in the context of Project Objectives Increase off-farm employment Support the process of strengthening self-government and regional development Assist in building institutional potential necessary to absorb EU pre-accession and structural funds Increase the level of effectiveness and efficiency of local and regional administration in creating nonfarm employment in rural areas Pilot Institutional Development Program (IDP) Technical assistance To MIAPA All-Polish Training Program IDP outcomes dissemination program Source: Consultant’s own analyses The objectives of Sub-Component B3 complied with the RDP assumptions and with the National Strategy for Rural and Agricultural Development, which assumed undertaking activities aimed at improvement of sustainable development conditions in rural areas, development of social and technical infrastructure intended to enhance the attractiveness of investing in rural areas and supporting the creation of non-farm employment in rural 105 areas . When analysing the purpose of RDP and its analysis from the beneficiaries’ point of view, one should state that the opportunity to provide additional computer equipment to offices and to introduce improvements that would enable better absorption of structural funds was most important for them. The changes they decided to make did not result in a total rearrangement of the institution, nor to replacement of the bureaucratic management paradigm with the managerial one, as it was specified in Project expected outcomes. At the same time, the self-government administration has not become a real promoter of entrepreneurship in rural areas through changing legislation, nor has it found its place in the process of creating new jobs. Similarly, the Training Program fits in the overall assumptions of RDP, as it was supposed to strengthen the potential of selected territorial self-government units in the field of institutional development projects management and thereby to prepare them to absorb EU pre-accession and structural funds. These intentions, in the context of SAPARD program and Sector Operational Programs being launched now, seem to indicate the Training Program authors’ long-term thinking. When analysing Project documents, especially the final report of Sub-Component B3 implementation, one may notice some discrepancies in the formulation of objectives of Sub-Component B3 which was defined as increasing the level of management effectiveness and efficiency in units covered by the RDP B3 through strengthening the institutional potential and through training. The long-term objective of Sub-Component B3 implementation was to contribute to the creation of institutional conditions for continuous 105 National Plan for Regional Development of the Ministry of Regional Development and Building. Warsaw 2000, p.18. 218 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka improvement of management in local and regional public administration, including rural areas. The so defined objective does not specify the purpose of increasing management efficiency in territorial self-government units. The definition of RPD objectives envisaged that achievement of this objective would improve the starting conditions for creation of offfarm employment in rural areas. Thus, one may agree with Tomasz Potkański that Sub-Component B3 “was implemented ‘by the way’, on the occasion of the rural development project (i.e. in the areas outside the main cities) and that it served this purpose genuinely, but it was a purpose for itself – it was intended to build institutional conditions for gradual improvement of management 106 quality . Taking the so-defined overall objective, the managers of Sub-Component B3 specified detailed goals, calling them “strategic goals”. The first strategic goal A was defined as: the existence of a system disseminating the experience concerning the method of implementing the Institutional Development, intended for self-governments and their organisations, advisory and training institutions, universities educating staff for selfgovernments; strategic goal B was defined as a legal system (legal surrounding) conducive to enhancement of management quality in territorial self-government units; strategic goal C was the existence of government and self-government institutions’ agreement concerning development of a system enhancing the quality of management in 107 self-governments . The following activities were expected to achieve the goals listed above: • Institutional Development Programs (IDPs) in pilot units intended to develop instruments for enhancing the efficiency of regional and local administration, including finance and human resources management; • Training programs aimed at improving the level of services provided by members of local authorities and by public administration officers and at building the potential for implementation of EU rules and regulations. Here, training was supposed to be organised for selected members of self-government activities, i.e. elected councillors, board members and personnel of various levels of self-government administration. Particular focus was to be on public management and administration services significant in terms of successful implementation of RDP and its Components. • Capacity building program aimed at the Ministry of Interior and Public Administration in the area of ability to co-ordinate IDP and training activities and to prepare analyses and studies. Here, training for selected members of self-government authorities was to be organised, i.e. for elected councillors and board members and personnel working at various levels of self-government administration. Special focus was on public management and administration services for successful implementation of the RDP and its Components. 106 Tomasz Potkański, Jacek Nowak. Metoda Rozwoju Instytucjonalnego jednostek administracji publicznej, przygotowany i wdrożony w ramach projektu MSWiA” Budowanie potencjału instytucjonalnego administracji lokalnej i regionalnej” (The Public Administration Units Institutional Development Method Prepared and Implemented under the Project of MIAPA Building the Institutional Potential of Local and Regional Administration), Samorząd Terytorialny, Year XX January - February 2002; p.36. 107 Ibidem, p.8. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 219 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Institutional Development Program IDPs had a relatively limited coverage – 33 territorial self-government units (30 STU and 3 voivodship administration offices). According to program assumptions, this activity was envisaged as an experimental field for developing new management methods for selfgovernment administration. IDP participants were selected in a two-stage competition of territorial self-government units where voivodships were selected first and thereafter – TSUs recruited from the selected voivodships. The competition procedure was designed to select units well prepared to undertake the management improvement process and having some experience and their own achievements in this field. Checking, whether the managing team members were willing to improve the performance of their institutions and to become leaders of changes was one of the selection criteria108. Interviews made with participants of the Institutional Development Program enabled us to state that the self-government units’ capacity to mobilise themselves to undertake joint tasks was the basic selection determinant. The units had to communicate within the frames of one powiat and to communicate with Powiat Starosta Office in order to undertake joint activities. Many of the beneficiaries participated in IDP, as they regarded it as a chance to provide additional computer equipment for their offices and to be able to meet the requirements imposed by the regulations governing the procedure of applying for structural funds. According to many of the participants, strategic plans, long-term financial plans and longterm investment plans used for the preparation of the Local Development Plans were the most useful results of the whole program. At the same time, the beneficiaries stressed that most of the activities were planned to improve the living conditions and to perform local undertakings first of all. Many of these undertakings contributed to the better assurance of civilisation needs first of all, and not as much to enhancing the economic competitiveness of territorial self-government units. Dissemination Program The Dissemination Program target group included all 2792 Polish gmina and powiat selfgovernments. 1225 self-government units participated in the Program and the remaining 1567 self-government units who did not attend the dissemination conference, were sent printed materials and software on CDs. 823 self-government units participated in workshops, e.g. on ethics and counteracting corruption and the remaining 1969 who did not accept the invitation, were sent manuals and training materials. All-Poland Training Program The participation in the all-Poland training program was of a mass nature – the program covered 642 self-government units (4555 officers and councillors attended the training). Evaluation of participation According to Project documents, primary beneficiaries of the RDP were defined as rural residents those served by public utilities (recipients of new and improved services), civil servants (recipients of training and better systems management), unemployed persons and former state farm employees (recipients of training and greater job opportunities), new and existing entrepreneurs (recipients of credit, technical assistance, training), parents and children (recipients of better quality schools), teachers and local 108 220 Ibidem, p.10. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka administrators (recipients of training and an improved teaching environment), and women. In terms of Sub-Component B3, representatives of self-government administration were the most important participants. Territorial self-government units were expected to play an important role in success of RDP through activities of Sub-Component B3 – the Institutional Development Program and the All-Poland Training Program for officers of territorial self-government units. Moreover, Sub-Component B3 was intended to increase the efficiency and effectiveness of Project activities, especially of Component C – rural infrastructure and of SubComponent B2 - education. Similarly significant tasks were ascribed to territorial selfgovernment units in implementation of the National Development Plan, which provides that “State’s regional policy should gradually support the growth of territorial self109 governments’ own funds and the rational expenditure of public funds by them” . It was expected that due to implementation of a rational policy of public finance management by territorial self-government units, they would generate more funds enabling them to increase the scale of investments in their area. Selection of IDP beneficiaries The IDP method was developed during the two years’ pilot project (2001-2003) implemented in 33 public administration units representing the Government administration in voivodships (Voivodship Administration Office in Krakow, Białystok and Szczecin), voivodship self-governments (Marszalek Offices in Krakow, Białystok and Szczecin), powiat self-governments (Powiat Starostwo in Myślenice, Łomża and Gryfino) and gmina self-governments (gmina administration offices) on the territories of these powiats. It was also considered that two additional powiats might participate in the Program: the powiat of Szczecinek from Zachodniopomorskie voivodship and the powiat of Nowy Sącz from Małopolskie. They had taken the vice-leaders’ positions in their regions as a result of evaluation, although the distance in points between them and the leaders were small in both cases. Finally, the following territorial self-government units were selected for IDP: • Małopolskie voivodship: Voivodship Administration in Krakow, City Administration in Krakow, Powiat Starostwo in Myślenice, City and Gmina Administration in Myślenice, City and Gmina Administration in Dobczyce, Gmina Administration in Pcim, Gmina Administration Raciechowice, Gmina Administration in Siepraw, City Administration in Sułkowice, Gmina Administration in Tokarnia, • Zachodniopomorskie voivodship: Voivodship Administration in Szczecin, City Administration in Szczecin, Powiat Starostwo in Gryfin, City Administration in Banie, City Administration in Cedynia, City and Gmina Administration in Gryfino, City Administration in Chojna, City Administration in Moryń, Gmina Administration in Stare Czarnowo, City Administration in Trzcińsko Zdrój, Gmina Administration in Widuchowa, • Podlaskie voivodship: Voivodship Administration in Białystok, City Administration in Białystok, Powiat Starostwo in Łomża, Gmina Administration in Łomża, Gmina Administration in Miastkowo, Gmina Administration in Nowogród, Gmina Administration in Piątnica, Gmina Administration in Przytuły, Gmina Administration in 109 NPR, p.22. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 221 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Śniadowo, Gmina Administration in Wizna, Gmina Administration in Zbójna, City Administration in Jedwabne. In the context of entire Sub-Component B3, the selection for IDP-related activities seemed particularly significant. The results of these activities were to be disseminated in territorial self-government units and consequently – to contribute to achievement of a sustainable effect of RDP activities. The correct selection of territorial self-government units for Pilot IDP implementation was expected to contribute to the success of dissemination measured in the number of territorial self-government units who implemented the proposed solutions. The experience gained under previous projects of institutional transformations implemented under PHARE, DIFID, USAID showed that dissemination of changes implemented in pilot units had become the key problem. According to this experience, projects implemented in pilot units were of a “bumper crop” nature and were disseminated in other TSUs with great difficulty. But analysing the social and economic conditions and the nature of units selected for implementation of the Pilot IDP one should state that they were differentiated and this fact may contribute to the better dissemination of outcomes, with a reservation however, that there was an overrepresentation of TSUs situated in metropolitan areas110. In the future, this may be referred to as an argument by those reluctant to adopt institutional solutions in other gminas, especially in those of a typically rural nature, particularly that the “different conditions” or “inadequacy to conditions” in which a given TSU functions was one of the causes of difficulties in dissemination of institutional transformations indicated by the respondents – TSU representatives participating in regional dissemination conferences. It is hard to state, to what degree the statements made by TSU managers resulted from their analysis of innovations proposed to them, and to what degree they were a manifestation of their reluctance to any changes. Selection of participants for the All-Poland Training Program 642 self-government units participated in the all-Poland training program (4555 officers and councillors in total). The increase of participating self-governments’ potential in the field of project management should have been ensured through inviting teams of territorial self-government units consisting of representatives of three groups responsible for making decisions in self-government units: councillors, board representatives and key officers. Territorial self-government units were recruited by Regional Steering Committees in voivodships. Territorial self-government units applying to the training program were supposed to submit an application form presenting the problem they wanted to solve by the means of preparing and implementing a project. According to the Training Program rules, training should be attended by teams from territorial self-government units and not by individual representatives of gminas, powiats or voivodships and that training workshops, besides providing information and knowledge from various disciplines relating to public administration, should lead to the integration of teams from territorial self- 110 Urszula Klimska, Paweł Swianiewicz: Społeczne i polityczne zróżnicowanie aglomeracji w Polsce – waniliowe centrum, mozaika przedmieść. Prace Geograficzne tom XX. Wydział Geografii i Studiów Regionalnych. Warszawa 2005 por: Obszary metropolitalne w aktualizacji przestrzennego zagospodarowania kraju w: Materials from conference „Metropolitan Network as a Binder of Poland and Europe Development. Polish Metropolies in the National Development Plan. Warsaw 2829.04.2005 see: K.Trafas, A.Zborowski Krakowski Obszar Metropolitalny – Metropolitalne Funkcje Krakowa. A study for Marszalek Office, Krakow 2003: G. Gorzelak, 2002: Polskie regiony w procesie integracji europejskiej, in: Studia Regionalne i Lokalne, no 2-3, and also M. Smetkowski, 2001: Nowe relacje miedzy metropolią a regionem w gospodarce informacyjnej, in: Studia regionalne i lokalne, no 4. 222 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka government units and that as a result of their work a project enabling the to solve the indicated problem should be prepared.111. The strategy of recruitment was based on criteria specified by the Regional Steering Committees. The Committees adopted various approaches here. Some RSCs applied the rule of selecting the most promising teams of territorial self-government units, while other selected territorial self-government units with the weakest economic condition, assuming that participation in the Program might strengthen the. Another strategy accepted the basic determinant in the form of clearly outlined strategies of territorial self-government units’ co-operation with local partners. 1063 teams from territorial self-government units reported to the Program in total, i.e. nearly twice as much as the number of places the National Steering Committee had envisaged (549). 636 territorial self-government units / 4195 qualified for training activities organised by RSC/VPIU (in co-operation with the Foundation for Local Democracy Development): • 503 teams from gmina administrations or gmina and city administrations; • 131 powiat teams represented by personnel of Starostwo (powiat council); • 2 teams from Marszalek offices (Pomorskie and Warmińsko-Mazurskie voivodship). The fact that there were so few teams from Marszalek offices does not mean that these institutions were not interested in the Program. City Administrations from many voivodships did not delegate their teams (wishing to leave as many places for gminas and powiats as possible), but only their single officers who became members of gmina and powiat teams. During the Program, 41 territorial self-government units, i.e. about 6.5% resigned. Lack of time, inability to delegate a team consisting of several persons or the fact of not preparing and not submitting their project in due time were the most typical reasons of withdrawal. An ex-post analysis of characteristics shows that “in case of most voivodships, the RDP Training Program was attended by territorial self-government units with a low population density; in some cases the regional characteristics resulting from the settlement pattern or from natural conditions caused that the density of population was higher in territorial self-government units participating in RDP. As it had been assumed, territorial selfgovernment units of a rural nature were prevailing in the Program. (...) In most cases, they had a weak development potential, were not much attractive and were threatened with depopulation”112. Thus, one may agree with the authors of the report who state that the Program reached the recipients it had been intended for. Situation prior to and after Implementation of Sub-Component B3 Today, it is difficult to reconstruct the territorial self-government units’ motivation to participate in the Pilot IDP. When carrying out the evaluation survey, it was hard to find persons who had made the decision in some cases, as they were not re-elected for the function of wójt, mayor or starosta or the did not work in the given territorial self- 111 112 Final Report of RDP Training Program Implementation, Component B3, p.2. Final Report of RDP Training Program Implementation, Component B3, p.2. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 223 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka government unit any longer. In those territorial self-government units where the decisionmakers were re-elected for the function of wójt, mayor or starosta, the motivation behind participation in the Program is perceived in terms of its achievements now. Many of the respondents stated that they had been motivated to participate the pilot program by the wish to enhance their gmina management, but they also admitted that also by the wish to receive computer equipment. As the respondents stressed, they had not entirely understood the purpose of the pilot program after the first meetings. The fact that computer equipment would be provided had been the only thing clear to everyone. As a result of further meetings with consultants and program organisers, the motivation to implement the planned transformations was growing. Should one assess the declared motivation to participate in the project on the basis of continuation of changes initiated under IDP, it should be called into question. There were only few cases when leaders of territorial self-government units continued implementation of further changes proposed under IDP. When asked, why they implemented these changes, they stated that the implementation of these tools (e.g. the self-government officer code of ethics, the councillor code of ethics, job descriptions) was a requirement of participation in “Transparent Gmina” competition organised by Gazeta Wyborcza daily newspaper. In one case, a respondent wished to repeat the questionnaire survey in which gmina inhabitants had assessed the work of wójt, in order to acquire information about the support he might rely on. The remaining respondents, when asked why they limited themselves to changes implemented under IDP stated that they did not continue the innovations, as these were inadequate to their unit’s needs. They also indicated that the model solutions proposed in the program were too “theoretical” and were not adjusted to implementation in their territorial self-government unit. It is therefore difficult to state, to what degree the territorial self-government units leaders could see the need to introduce changes and whether a need like this existed. Almost all of the respondents confirmed that prior to participation in the Pilot IDP they had not been aware of the need to make any improvements in their institutions’ work. No sooner than in the phase of self-assessment they realised that changes recommended by IDP experts should have been introduced. It seems that none of the respondents saw the need to change the overall approach to the self-government administration management in a manner making it closer to the New Public Management model. It seems that it is not the wish to be in the same league with the best managers, but the necessity to adapt to new legal regulations that is the basic motivation to implement changes. This approach to changes in management may explain the selection of tools offered to territorial self-government units under the Pilot IDP. IDP resulted first of all in preparation of the strategic plan, the long-term investment plan, the long-term financial plan and operational plans prepared as a supplement to the strategic plan. No procedures replacing the bureaucratic model with the managerial model of management were implemented in these gminas. A similar approach was adopted when drawing up a catalogue of tasks which was incorporated in the Public Information Bulletins prepared by gminas. 224 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka In case of IDP dissemination program, it is difficult to find out now, what the motivation of participants of regional conferences disseminating IDP was and to what degree their needs have been satisfied. The conference organiser was not obliged to carry out the conference evaluation in terms of needs, quality of lecturers or motivation. Participants of regional conferences attended these events, because they were organised by Marszalek Office and the Ministry of Interior and Public Administration. This was the main motivation behind territorial self-government units representatives’ participation in regional conferences. Their opinion was that if they were invited by institutions so important in the Project, some decisions might be made during the conferences that would apply to RDP implementation or accounts directly. The participants did not expect that they would be given any materials or that they would learn anything interesting about territorial selfgovernment units management during the conference. The fact that in many cases the personnel of lower levels, or secretaries of territorial self-government units were sent to stand in for wójts, city presidents and starostas who had been invited also gives evidence to the attitude to disseminating conferences. It was also difficult to define, to what degree the conference met the expectations, as most of the respondents who had attended it were not able to reconstruct the course and the contents of the conference. The materials113 distributed during the conferences were not much appreciated by the participants either. Most often, when asked whether these materials would be useful in improving their institutions performance, the respondents answered that they would be of a limited use, as they were too theoretical and they did not cover the special nature of rural gminas. When the survey was carried out, attention was paid to the role of Marszalek Offices and Powiat Starostwo in dissemination of various tools in gminas participating in the Pilot IDP. It seemed that the proximity and the fact of maintaining day-to-day contacts would cause the good practice to become common faster. The survey showed no case of experience transfer from participating gminas to others that had not been covered by the pilot program. Participants of the All-Poland Training Program wanted to acquire knowledge about the opportunities and the methods of obtaining funds first of all. Some representatives of institutions were also seeking a chance for improving the performance of gmina and its institutions. 113 In the first phase of the survey it turned out that the respondents attending the conference had problems with identifying the materials they had been given. In order to obtain any reliable information, first pages of conference materials were e-mailed in the second phase of the survey, to show the respondents what the interview would be about. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 225 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 40. Main reasons why territorial self-government units decided to participate in the Training Program Motivation Attendance Proportion Access to funds 8 14.8 Knowledge about opportunities and methods of obtaining funds 44 81.5 Counteracting the unemployment, activation 5 9.3 Improving gmina and institution performance 13 24.1 Exchange of experience and contacts 7 13.0 Other reasons 4 7.4 Hard to say 3 5.6 Total 54 100.0 Note: the proportions do not total to 100% - the respondent was allowed to choose more than one answer Source: Consultant’s own analyses Effectiveness evaluation The effectiveness of Sub-Component B3 will be evaluated in two aspects. The first aspect covers specific activities ascribed to Sub-Component B3 – the Institutional Development Program, the All-Poland Training Program and the Dissemination Program. The second aspect covers the effectiveness of activities undertaken within the frames of Sub-Component B3, but related to co-ordination of Components A, C and Sub-Components B1, B2. In the evaluator’s opinion, it was assumed that the tasks of Sub-Component B3 “will facilitate the link between the IDPs and the training programs, foster the co-ordination with other IDP Components, and provide quality control as well as monitoring and evaluation services”114. Effectiveness of the Institutional Development Program The purpose of IDP was to develop and to test in practice a method of analysing, planning and implementing the Institutional Development undertakings (the so-called ID method), i.e. unaided improvement of management115. Project managers stressed clearly that “the implementation of management improvements is not a purpose for its own, but is intended to enhance the quality of services provided to individual and entrepreneurs and to establish the standards desired for these services”. When evaluating the effectiveness of IDP one should consider the issue from the following points of view: • effectiveness of ID tools development, • effectiveness of the implemented tools at the level of territorial self-government units implementing ID with the purpose of achieving IDP objectives, • effectiveness of the implemented tools in terms of RDP objectives achievement. 114 PAD, p. 68, point 109. Tomasz Potkański, Project Manager Final Report “Building the Institutional Potential of Local and Regional Administration”, p. 17. 115 226 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka To evaluate the effectiveness in terms of ID tools development one should take a look at the following figures: ID methods were tested in 33 pilot units where 217 tasks (projects) were implemented in total. This number included 157 tasks of a advisory and implementation-related nature and 60 supplementary training tasks. In effect, out of 37 recommended solutions only 11 items were recorded in the good practice catalogue, beside 26 that had been recorded earlier – for the illustration purpose and as a help in the implementation phase of pilot activities. When evaluating the effectiveness of projects implemented under the IDP from the territorial units’ point of view, one should state that in cases where the applied tools had some utility value, addressing the requirements of IOPRD or regulatory requirements, their effectiveness was very high. Gminas using these tools were more effective in obtaining EU funds than other gminas. It is difficult to evaluate the IDP effectiveness in terms of services quality. Leaders of territorial self-government units stated that in general clients were satisfied with the services provided by their institutions. But when asked, how they measured the quality of services, they were unable to quote, how much the quality (i.e. IDP effectiveness) had increased. According to results of evaluation surveys carried out among inhabitants of gminas covered and not covered by IDP, there differences in assessment of the quality of services provided by TSU one year after Program completion did not differ much – almost all of them ranged from 3 (the aspect has did not changed) and 4 (the aspect has rather improved). One may be tempted to make a statement that the outcomes were better in gminas included in IDP, but the scale is small – from 0.3 to 0.05. What is worth stressing however, the comparison of marks of TSUs included and not included in IDP in specific categories shows that each of these categories is always graded slightly higher in the units participating in IDP. The only distinct difference was found in availability of information provided by the phone – in this case the positive assessment favoured gminas covered by the IDP, where the average marks ranged from 4 (the aspect has rather improved) to 5 (the aspect has much improved). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 227 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka How quickly were the services provided 3.62 46.0 2. Clearness and comprehensiveness of information provided by servants 3.66 46.0 3. Comprehensiveness of rules used to provide services 3.61 44.2 4. Servants’ problem 3.5 48.9 5. Politeness of servants 3.81 55.1 6. Availability of information by the phone 4.48 64.2 7. Availability of places where one can sit and wait for services in comfort 3.92 50.1 8. Institution’s suitable working hours 3.89 47.0 9. Servants’ assistance in filling documents, writing applications, etc. 3.82 47.8 10. Servants’ honesty 3.79 55.8 11. General satisfaction with institution’s performance 3.7 53.0 interested in client’s the Change noticed in TSU not covered by IDP (% of resp.) 1. It. Average mark for gminas not covered by IDP Category Change noticed in TSU covered by IDP (% of resp.) Assessment of territorial self-government units performance by inhabitants of gminas covered and not covered by the IDP Average mark for gminas covered by IDP Table 41. 3.49 37.0 3.61 30.0 3.56 27.0 3.38 32.0 3.79 34.1 3.56 42.3 3.6 24.4 3.8 21.1 3.5 31.2 3.64 32.0 3.59 29.4 Source: Consultant’s own analyses It is interesting however that although in citizens’ subjective opinion, the quality of services is assessed almost identically in all categories specified in the questionnaire, the subjective perception of the change in the quality institution’s work within the last two years is definitely in favour of the units covered by the IDP. The change is clearly perceived in the categories of services quality improvement. 228 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 42. Improvement of territorial self-government units’ performance in the last two years in the opinion of citizens in gminas covered and not covered by the IDP (% of responses) It. For units covered by IDP Category were Definitely improved Rather improved Definitely improved 31.8 4.0 12.0 1.4 How quickly provided 2. Clearness and comprehensiveness of information provided by servants 33.6 4.7 11.0 1.1 3. Comprehensiveness of rules used to provide services 31.8 3.6 10.0 8.2 4. Servants’ problem 32.8 5.1 13.0 1.8 5. Politeness of servants 39.1 2.5 18.6 2.5 6. Availability phone 29.9 5.5 12.0 1.1 7. Availability of places where one can sit and wait for services in comfort 40.4 11.3 9.7 1.4 8. Institution’s suitable working hours 35.0 4.7 7.5 1.8 9. Servants’ assistance in filling documents, writing applications, etc. 32.8 5.1 11.5 1.4 10. Servants’ honesty 27.4 6.2 12.5 0.7 11. General satisfaction institution’s performance 39.8 5.8 14.7 1.1 of services Rather improved 1. interested the For units not covered by ODP in information client’s by with the the Source: Consultant’s own analyses Differences in assessment of the quality of institutions’ performance are much more visible in the group of micro-entrepreneurs operating in gminas covered and not covered by the IDP. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 229 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 43. Assessment of territorial self-government units performance by micro-entrepreneurs (average marks) It. Category Average mark for gminas covered by IDP Average mark for gminas not covered by IDP 1. How quickly were the services provided 4.03 3.27 2. Servants’ competence 3.8 3.47 3. Clearness and comprehensiveness of information provided by servants 3.93 3.6 4. Comprehensiveness of rules used to provide services 3.97 3.43 5. Servants’ interested in client’s problem 4.13 3.47 6. Politeness of servants 4.37 3.93 7. Availability of information by the phone 4.10 3.69 8. Availability of places where one can sit and wait for services in comfort 4.33 3.33 9. Institution’s suitable working hours 4.17 3.87 10. Servants’ assistance applications, etc. 4.32 3.45 11. Servants’ honesty 4.48 3.93 12. General satisfaction with the institution’s performance 4.2 3.57 in filling documents, writing Source: Consultant’s own analyses The effectiveness of objectives achievement should be considered very high at the level of IDP implementation in gminas. On the other hand, it is hard to assess the effectiveness of IDP in terms of RDP overall objectives achievement. Most of those implementing the project did not associate participation in IDP with the RDP objectives. Many of IDP managers and authorities of territorial self-government units did not know the RDP objectives. Effectiveness of the All-Poland Training Program On the other hand, most participants of the Training Program (83%) are of the opinion that their attendance contributed to improvement of their institution functioning – first of all through newly acquired skills of raising funds and writing projects (66.7% of respondents) and through knowledge, experience and better organisation of work. More than a half of the respondents (59.3%) were also of the opinion that their participation in the Program contributed to their better understanding of self-government administration functioning in the context of integration with the EU. Bearing these outcomes in mind one may state, that the objectives of the Training Program have been achieved. Effectiveness of the Dissemination Program The purpose of 16 conferences organised in capitals of all regions, for all gmina and powiat self-governments, was to disseminate the achievements of IDP. Assumptions, methods and basic tools developed in the pilot phase were presented during the conferences. Beside the theoretical part, self-governments participating in the Pilot IDP presented their practical experience. A’’ gmina and powiat self-governments in each of 16 voivodships were invited to attend the conferences. Eventually, 1225 self-governments (i.e. 46% of all gminas and powiats) participated in one-day conferences all over Poland. 230 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Number of participants to the overall number of TSUs in voivodship All participants of the conference 66 76 66 82 71 90 TSUs who accepted the invitation to the overall number of TSUs [%] Number of TSUs participating in the conference 76 Number of TSUs who accepted the invitation 115 Overall number of TSUs Voivodship świętokrzyskie Number of conference participants from TSUs 01.04.2004 TSUs participating in conference to the overall number of TSUs [%] Dissemination of the achievements of IDP Date of conference Table 44. 02.04.2004 opolskie 82 42 51 42 51 52 63 60 15.04.2004 śląskie 184 75 41 68 37 76 41 84 16.04.2004 łódzkie 198 88 44 63 32 70 35 78 22.04.2004 małopolskie 201 106 53 93 46 106 53 114 23.04.2004 podkarpackie 180 101 56 79 44 88 49 96 06.05.2004 podlaskie 135 78 58 53 39 58 44 64 07.05.2004 warmińskomazurskie 132 67 46 67 51 72 55 79 20.05.2004 lubelskie 233 145 62 113 49 115 49 123 21.05.2004 mazowieckie 351 157 45 107 31 121 34 142 27.05.2004 pomorskie 139 80 58 67 48 76 55 88 28.05.2004 kujawsko-pomorskie 163 103 63 101 62 102 63 111 03.06.2004 zachodniopomorskie 132 76 58 66 50 70 53 78 04.06.2004 wielkopolskie 257 145 56 103 40 107 42 118 17.06.2004 lubuskie 95 57 60 45 47 47 50 58 18.06.2004 dolnośląskie 195 122 63 82 42 89 47 99 2792 1518 54 1225 44 1301 47 1480 Total Source: report of the Union of Polish Cities and the Polish Union of Rural Gminas, as of 30.06.2004 As a rule, regional conferences were attended by the invited representatives of authorities – wójts, mayors or starostas, sometimes heads of departments and in some cases even rank and file employees. In the opinion of territorial self-government units representatives, numerous invitations for various conferences and training events make it impossible to attend all of them and this is why they often send someone to stand in for them. The conclusion is that participation in the conference was formal and that it did not translate into any direct use of the knowledge and the materials to make improvements in territorial self-government units. It was hard to find the conference materials in many of the institutions visited during the evaluation process. Consequently, it was impossible to assess the solutions that had been presented. And even if the territorial self-government units staff had explored them, these solution could not find any direct application in their gminas according to them. In their opinions, the solutions were too theoretical and could not solve the problems representatives of territorial self-government units’ authorities and managers encounter in their day-to-day work. The assessment of the “Local Self-Government Management Leader” competition effectiveness was another element of the dissemination program evaluation. The purpose Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 231 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka of the competition was to disseminate the management solutions among other territorial self-government units in Poland. Dissemination of solutions selected as a result of the competition was expected to contribute to improvement of the quality of services provided to citizens and business entities by the self-government administration.116 72 projects were submitted and the Competition Commission selected 24 finalists out of them and thereafter – 8 winners. It is difficult to assess the effectiveness of this activity in terms of IDP objectives achievement. There are no data available about the number of territorial self-government units who used the good practices described in materials distributed during the regional conferences and on www.dobrepraktyki.pl. site. In the course of interviewing IDP participants, an attempt was made to find out how interesting the implemented solutions had been to other TSUs. The conclusion was, that only few persons talked to them during the dissemination conferences and almost no one – after the conferences. The dissemination program included also workshops on ethics and counteracting corruption. The purpose of the workshops was to equip self-governments with tools of consistent and systematic building the image of a transparent institution which communicates with the citizens, with particular focus on implementation of procedures, codes of ethics, development of controls. 823 self-governments accepted the invitation and attended the workshops, this group accounting for nearly 30% of all selfgovernments. 2143 persons were trained in total. The remaining1969 self-governments who had not confirmed their intention to attend, were sent training materials by mail. The self-government press published a number of articles promoting the problems of ethics in territorial self-government. This is another task where it is difficult to evaluate the effectiveness. Evaluation of Sub-Component B3 utility Utility of IDP The purpose of IDP was to test the method in 33 pilot units. 217 tasks (projects) were implemented under the Program. The intention of IDP authors was to enable these units to continue improving management on their own. It was stressed at the same time, that the implemented management improvements would not be a purpose for itself, but that they would result in improved quality of services provided to inhabitants and entrepreneurs and in establishing and maintaining the desirable standards of these services. Thus, the utility of the Program should be analysed from two points of view: the utility of IDP for participating gminas and the utility of IDP dissemination program. The units participating in IDP assessed its utility very well. Particularly high marks were given to the utility of tools, the implementation of which related to the territorial self-government units’ obligation to observe regulatory requirements. It was stressed that the catalogue of services was particularly useful in preparing the Public Information Bulletin. High marks were also given to the utility of activities related to requirements imposed by European Funds. Especially, the utility of strategic plans, long-term investment plans, long-term financial plans and operational plans for financial plans developed under the Pilot IDP was emphasised. Moreover, high marks were given to implementation of procedures facilitating creation of projects and execution of investments in territorial self-government units. At the same time, the respondents indicated the low utility of such tools as surveying inhabitants of rural areas, task-oriented budget or programs of co-operation 116 see: Project Manager’s Final Report “Building Institutional Capacity in Local and Regional Administration”, Tomasz Potkański, Warsaw 15.08.2004, p. 23. 232 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka between the mom-government sector with territorial self-government units. IDP participants covered by the survey indicated these tools’ inadequacy to the reality of rural gminas and regulatory requirements concerning budget construction. Most self-governments adopted a very practical approach to the activities proposed by the program. Participants of IDP were interested in preparing documents needed to obtain pre-accession and structural funds – this thesis is confirmed by the selection of tools implemented in IDP gminas. When evaluating the utility of Sub-Component B3 for the dissemination program, one should state that only 11 out of 37 tools were incorporated in the catalogue of good practices recommended by RDP. The limited selection of tools included in the dissemination program resulted from restrictions imposed By MIAPA. Having asked external evaluators for their opinion it had been found that the good practice base contained too many projects implemented under IDP and considering that their life-time was too short, a decision was made to include only 11 practices in the base. The set of good practices was supplemented with the solutions awarded in the “Local Selfgovernment Management Leader”. It is a problematic thing to evaluate the utility of publications, as these are not known to the interviewed participants of the Pilot IDP. The respondents use the materials published on MIAPA www sites very seldom. When they do, they usually look for new legal regulations and respective explanatory materials. But it is impossible to obtain any objective assessment of the degree of interest in these materials, as no counter of visitors was installed. Utility of the All-Poland Training Program The Training Program utility can be evaluated very highly, as a vast majority (96.3%) of its participants who were surveyed indicated that the knowledge of project development methodology provided under the Program was sufficient (absolutely sufficient – 14.8%, rather sufficient – 81.5%). The respondents – members of teams preparing projects under the Training Program – are implementing their projects now (40.7%) or are planning to do this in the nearest time (53.7%). This shows that a definite majority of projects has a chance for implementation. Most often, these projects were submitted to SAPARD and IOPRD programs. As a result, according to their authors’ estimations, every third project can generate new business entities, while almost every second project can create new jobs in the off-farm sector. Most of the projects covered issues from the field of infrastructure development and environment protection management. 345 projects dealing with these issues were prepared, accounting for 58% of the overall number of projects. The group of problems covered by the projects included: water supply and sewage disposal management in gmina or region; waste disposal; modernisation of regional and local roads. The second place was taken by projects dealing with active policy of counteracting the unemployment. This group included 124 projects, i.e. 21% of the total number. They envisaged creating activation centres for the unemployed, creating loan guarantee funds, organising consultancy and education centres preparing the unemployed for employment and business activity in the conditions resulting from Poland’s accession to EU, occupational activation of unemployed women, creating job seeking clubs in gminas or powiats, improving the system of education for school graduates and the unemployed, etc. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 233 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka The third position was taken by projects supporting the private sector development. 55 projects were prepared here, i.e. 10% of all projects. They dealt with creation of gmina and powiat information centres or training and advisory centres for entrepreneurs, also in the aspect of Poland’s accession to EU; creation of local economic development centres or business initiatives centres; creating banks of offers for investors; supporting development of agri-tourism, etc. The next group of projects covered the education system management issues. They partly fit in Sub-Component B2, as they dealt with similar problems of teachers’ professional development. These projects – there were 35 of them – accounted for nearly 6% of all projects that were prepared. Apart from professional development issues, some projects covered also: education in the labour market context; rationalisation of the education system organisation, management and financing; obtaining funds for didactic aids; creating education strategies for gminas, etc. The fact that only 30 projects, i.e. less than 5% applied to the territorial self-government units organisation directly. These project focussed on issues related to creating client service offices/points, i.e. building client-friendly institutions; improving systems of information flow between the institution and the client; enhancing the quality of services provided to the local community (servant – client relation); implementing the quality management system in the institution, etc. Such selection of projects confirms the thesis which has already been formulated, that Program participants were first of all oriented towards acquiring knowledge about the selfgovernment administration’s current tasks, i.e. as efficient utilisation of support from preaccession and structural funds, instead of improving administration’s performance according to the New Public Management principles. Table 45. What did the All-Poland Training Program participants expected of RDP implementation? Proportion Obtaining funds for investments, financial support 48.4 Skills and knowledge concerning EU funds 53.2 Elimination of unemployment; development of entrepreneurship 9.7 Other 35.5 Do not remember / hard to say 3.2 Source: Consultant’s survey Moreover, most of the respondents declared also that projects developed under the Training Program had been envisaged in Gmina Development Strategy (63%), they complied with the Local Development Program (83.3%) and the Long-Term Investment Program (64.8%). One should notice here, how high the training program utility was in terms of the number of applications submitted. The respondents indicated that 68% of applications were submitted as applications for pre-accession or structural funds. It seems that most of the applications that had not been submitted covered the issues for which no co-financing had been envisaged under pre-accession programs and structural funds. 234 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 46. Projects of territorial self-government units worked out under the All-Poland Training Program What program did gmina / powiat / marszalek office apply to with the project prepared under the All-Poland Training Program? Proportion SAPARD 35.5 PHARE 3.2 IOPRD 35.5 SPO ROL “ Food Sector Restructuring and Modernisation and Rural Areas Development” 4.8 Other 18 Gmina did not apply at all 21.0 Do not remember / do not know 11.3 Source: Consultant’s survey On the other hand, outputs produced for the purpose of the Training Program were given rather high marks by Program recipients. Marks given to particular training modules and workshops in terms of utility for tasks performed by the institution were high and their average ranged from 3.64 to 4.33. The participants gave highest marks to the following modules: • Project management; • Transparent and friendly gmina; • Financial and strategic management; and workshops: • Development of infrastructure and management in the field of environment protection; • Labour market – active policy of counteracting the unemployment; • Education system management. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 235 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 47. Average marks for utility of particular modules and workshops Training module Average mark Module 1: Co-operation between territorial self-government units 3.83 Module 2: Financial and strategic management 4.08 Module 3: Institution organisation and functioning 3.89 Module 4: Public services, including utilities 3.97 Module 5: Stimulating economic development 3.78 Module 6: Human resources management 3.97 Module 7: Social participation and social development 3.78 Module 8: Project management 4.33 Module 9: Transparent and friendly gmina 4.16 Workshops Development of infrastructure and management in the field of environment protection 4.24 Supporting the private sector development 3.64 Labour market – active policy of counteracting the unemployment 4.19 Education system management 4.13 Organisational development of a territorial self-government unit 3.92 Source: Consultant’s own analyses Similarly, marks given for the utility of training materials in institution’s day-to-day work by those who declared that they had had contact with the given publication and decided to assess it, were rather high. Two publications can be distinguished here in particular: 236 • Project management; Author: Piotr Szczęsny; • Development of infrastructure and management in the field of environment protection; Authors: Piotr Szczęsny, Mariusz Adamowicz; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 48. Average marks for materials utility Publication Average mark Module 2: Financial and strategic management; Author: Krzysztof Pakoński 3.74 Module 3: Institution organisation and functioning; Author: Wiesława Borczyk 3.70 Module 4: Public services, including utilities; Authors: Grzegorz Dziarski, Wojciech Kłosowski 3.88 Module 5: Stimulating economic development; Author: Ryszard Kamiński 3.69 Module 6: Human resources management; Author: Jacek Kowalski 3.76 Module 7: Social participation and social development; Authors: Jacek Kwiatkowski, Karolina Stawicka 3.80 Module 8: Project management; Author: Piotr Szczęsny 4.21 Module 9: Transparent and friendly gmina; Author: Marek Jefremienko 3.88 Program 1: Development of infrastructure and management in the field of environment protection; Authors: Piotr Szczęsny, Mariusz Adamowicz 4.15 Program 3: Supporting the private sector development; Author: Zbigniew Pluta 3.43 Program 4: Education system management; Author: Andrzej Szeniawski 3.57 Program 5: Organisational development of a territorial self-government unit; Author: Ryszard Wilczyński 3.79 Case study: Regional co-operation in Pomorskie voivodship – case study; Author: Emilia Małgorzata Gimpel 3.57 Source: Consultant’s own analyses Utility of the dissemination program The dissemination program utility should be analysed through measuring, to what degree the proposed management-related solutions were accepted and implemented by territorial self-government units participating in this activity. It was assumed that IDP should respond to territorial self-government units’ needs in the field of management. Due to implementation of the IDP and the dissemination program, the process of enhancing the quality and the efficiency of management in self-governments in Poland should accelerate. The utility of the program evaluated in terms of universality of the implemented solutions can be assessed as unsatisfactory, but one should be aware of the fact that although the outcomes of Sub-Component implementation are being evaluated in retrospect, relatively little time has passed yet. The interviews with participants of conference disseminating IDP outcomes show that in their opinion the tools that had been presented were inadequate to the conditions in which self-governments functioned. Most frequently it was pointed out that the Long-Term Financial Plans and the task-oriented budgets were useless in the form in which they had been presented in IDP materials. The respondents stressed that the situation in which territorial self-government units functioned caused that Long-term Investment Plans and Long-Term Financial Plans were a sort of a fiction. Territorial self-government units, wishing to increase their chances for financial support, submit many applications, changing Long-term Investment Plans and Long-Term Financial Plans each time. Depending on the source of financing, they change the parameters, so that they are most adequate to financing institutions’ expectations. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 237 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka The task-oriented budget is another example of how the proposed solutions are inadequate. In the opinion of territorial self-government units’ representatives, there are no grounds for it in the existing legal conditions. According to regulatory requirements, the classification budget applies. Although the task-oriented budget might be an excellent finance management tool, the legislative regulations constitute a serious barrier to its use and dissemination. Moreover, attention was drawn to the fact that the planned growth of services quality and efficiency had not been achieved in those territorial self-government units, where the task-oriented budget had been implemented. Considering that territorial self-government units have a limited capability to make decisions concerning their property, the management methods and the rules of rewarding teachers for example, using the task-oriented budget is an excessive luxury. Similarly, the respondent expressed their concerns about the utility of such solutions as job descriptions or questionnaires to be filled by citizens to assess the gmina administration’s performance. The utility of workshops dedicated to ethics and counteracting corruption is the next element of evaluation. The objective of this activity was to provide knowledge about the methods of identifying and counteracting corruption in territorial self-government units. As the workshop participants included in the survey sample stated, the utility of tools presented there was disputable and examples presented in training materials did not entirely reflect the reality of territorial self-government units, particularly – of those in rural areas. They also stressed that the implementation of the self-government administration code of ethics should be interrelated with changes in the self-government administration staff recruitment, to make it similar to those applicable to the civil servants corps. In the opinion of program participants, implementation of the code of ethics in gminas is an additional task which is not a priority for gmina council members or for wójts. Selfgovernment officer and gmina councillor codes of ethics were prepared in a few gminas, as recommended. While self-government officer codes were implemented without problems, those concerning council members did not generate in the circles of councillors. Should one analyse the effects of this year’s competition “Transparent Poland” organised for self-governments by Gazeta Wyborcza, the Polish Agency for Rural Development and the Foundation for Local Demicracy Development, where more than 800 self-government activists take part, it is characteristic that these selfgovernments implement self-government officer codes of ethics and councillor code of ethics separately, i.e. the tools that were developed and promoted under IDP. Thus, one may assume that we are facing a slow implementation of changes in the self-government environment’s organisational culture, the changes that will certainly be accelerated by the act on the self-government officer’s status which is being implemented now. 4.2 Bottom-up Analysis 4.2.1 Identification and Measuring Program Costs and other Outlays Sub-Component B3 Budget In case of Sub-Component B3, the costs did not go beyond the assumed budget. The budget was spent in 100%. Most funds of Sub-Component B3 budget, i.e. 38.61%, were allocated to the Institutional Development Program. If one adds the computer equipment intended to support the performance of IDP tasks, the dissemination program and the function of IDP specialist, 55.83% of the budget were planned to cover this undertaking. The next item of the budget – 33.66% – applied to the training program. The remaining funds were allocated for expert works performed by the reference team – 4.45% and for training specialists – 1.28%, short-term specialists – 0.48%, the project manager – 2.01% and the monitoring and evaluation specialist. In terms of the project budget, the 238 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Institutional Development Program implementation and dissemination of its results were the priority tasks. On the basis of agreement between MIAPA, MoF and MARDA dated 31.01.2004, amendments to the budget approved on 21.03.2001 were made. The new budget reduced the funds for the training program by 15%, for the Institutional Development Program by 12.7%, as not all funds were spent; for the individual consultant – training specialist’s fee by 25%, for the individual consultant – monitoring and evaluation specialist fee by 10%, for the for the individual consultant – IDP specialist by 13.3%. At the same time, the fund for the dissemination program was increased by 14.4%. The remaining items, such as costs of expert opinions prepared by the reference team, fees of short-term individual consultants and funds for purchasing computer equipment remained on the previous level117. As significant economies had been made in Sub-Component B3 budget, it was agreed with the World Bank (May 2003) that some part of the funds would be spent on purchasing computer equipment for MIAPA to strengthen DAP institutional potential and to continue the program after RDP closing. It was also agreed that a part of this equipment will be given to self-government organisations at no charge, to provide them technical equipment for continuation of the dissemination program. Some items were purchased as rewards for the “Local Self-Government Management Leader” competition.118 Costs incurred by the beneficiaries Financial costs incurred under the IDP covered mainly the beneficiaries’ “own cofinancing” of the purchased equipment. It was specified that this contribution could not be lower than 25%. Territorial self-government units were allowed to quote as their own contribution all investment outlays required to install the equipment i.e. refurbishing, purchasing additional computer equipment which had not been covered by the specification, laying computer network wires. In most cases, self-governments’ cofinancing was much higher than the compulsory 25% and reached the average level of 30% for all territorial self-government units. Some self-governments, such as Marszalek Office in Białystok, provided their co-financing at the level of 50%. Additionally, besides their work, members of project teams attended workshops and training organised in Powiat Starosta Offices. These institutions incurred additional costs here to provide transportation of training and workshop participants. Territorial selfgovernment units’ personnel members performed their IDP-related tasks within the frames of their job duties. In the opinion of territorial self-government units’ managers, these tasks did not represent any additional work load. They stated also, that participation in IDP had no negative effect on institutions’ work and that all tasks and services were carried out in due time. Neither did IDP implementation result in any worsening of the institution’s performance quality. Some participants of the program – members of working teams the evaluators managed to reach stated that program implementation had imposed additional load on them. They 117 Calculations were made on the basis of: MIAPA Project Team’s Financial anlythics, A. Skóraś, in: Project Manager’s Final Report: Building Institutional Capacity in Local and Regional Administration, Warsaw, 15.08.2004, p. 22. 118 Tomasz Potkański, Project Manager Final Report “Building the Institutional Potential of Local and Regional Administration” ;Warsaw 15.08.2004, p. 22. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 239 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka performed many of the tasks related to IDP implementation after their office hours. Implementation of the task-oriented budget, of Long-Term Financial Plans and LongTerm Investment Plans was particularly burdensome. The personnel was not paid any additional money for working overtime. Some of them stated that the result of the program implementation was not worth the effort. The promise of equipping their workplace with the new computer equipment purchased under the program was an antidote against employees’ lack of motivation. To increase personnel’s motivation, it was announced in Łomża that only those officers would be equipped with new computers who participated in the project. On the other hand, most participants of the Training Program did not identify any additional costs of their personal participation in the Program or any costs incurred by the institution. Financial leverage effect Apart from gminas’ commitment to the process of computerisation, no other additional involvement of funds was found out in activities of Sub-Component B3. Certainly, one may associate the applications to PHARE and SAPARD prepared by participants of SubComponent B3 within the frames of particulars tasks with the funds they managed to obtain. It is difficult and even impossible to prove any such relation though. No explicit data are available to show that the effectiveness of territorial self-government units participating in B3 activities in obtaining PHARE or SAPARD funds increased. Any statement like this could not be reliable when evaluating projects implemented under IOPRD, where not the application quality, but the amount of funds Marszalek Offices have at their disposal is the basis on which the financial support is granted. The Institutional Development program did not contribute to starting the financial leverage either. As a result of IDP, territorial self-government units did not acquire any additional, external funds. At the current stage of development of gminas participating in IDP it is difficult to assess, to what degree the Program contributed to increase of employment opportunity through creating more favourable conditions of business development. 4.2.2 Identification and measuring program outputs and outcomes Below, indicators of outcomes planned in the Project Appraisal Document and the Operational Manual are presented. Project Appraisal Document presents the performance indicators in Annex no 1, where Project construction is summed up and in Annex 2, where the Project is described in detail and where outcomes expected of particular Components are specified. 240 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 49. Performance indicators planned in the Project Appraisal Document and in the Operational Manual PAD – Annex 1 PAD – Annex 2 Operational Manual Number of decision makers and senior staff trained in selected administrative units for an average of 30 days of training About 3000 decision makers and senior staff in pilot and other administrative units trained for and average of 30 days of training per trainee The operational manual indicates in general that monitoring will have a form of outputs description (trained employees, new procedures, new personnel, etc.) Significant improvement in public services in about 20 pilot administrative units. It indicates also that Sub-Component B3 performance indicators have been specified as follows: - average number of training hours per employee - cost of training per one man-hour. It should be stressed that these are efficiency indicators – while in the remaining Components and in PAD – quantitative objectives have been specified. Source: based on Project documents Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 241 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 50. Assumptions for Sub-Component B3 monitoring Monitoring table System dissemination Institutional Number of training organisations offering management Development experience and courses to territorial self-government units: 3 implementation methods active. Number of consulting companies offering assistance to territorial self-government units in improving management systems with the tools prepared under the program: 7 Legal system conducive to enhancement of management quality – 2 recommendations will be prepared for each of 9 management areas. % of recommendations of legislative changes that have been implemented by the means of legal acts: 10 % of recommendations of legislative changes that have been introduced to draft laws and legal acts: 25 Agreement of government and self- A written agreement between self-government organisations government institutions concerning and a competent government agency (agencies): 2 creation of a system for management quality improvement in territorial self-government units Dissemination experience of program Number of territorial self-government units not participating in the program, whose representatives participated in the dissemination program: 600 Number of manuals distributed among territorial selfgovernment units not participating in the program: 2800 Institutional analysis, preparation Number of territorial self-government units for which the and implementation of institutional Institutional Analysis was prepared: 33 development plans (IDP) by Number of territorial self-government units which launched territorial self-government units IDP: 20 % of territorial self-government units participating in IDP, which improved the quality of management: 60 % of territorial self-government units participating in IDP, which improved the quality of services: 30 Training methodology developed: Yes/No and “distance learning” tools Preparation of training materials Number of trainers trained: 50 and staff of trainers Pilot training for territorial self- Number of government units training: 60 territorial self-government units attending units attending Number of persons trained: 420 Number of training man-days: 7 000 Training for territorial self- Number of territorial self-government government units’ representatives training: 540 in 9 management areas Number of training man-days: 63 000 Number of projects commenced: 360 the implementation of which Number of territorial self-government units participating in “distance learning” sessions actively: 300 Source: on the basis of Project documents Outcomes and outputs of IDP One of the indicators envisaged development of a method and procedures of institutional analysis and of IDP format by the end of the year 2001. 242 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka The number of territorial self-government units for which the institutional analysis was prepared was the next indicator. In this case, it had been planned that the institutional analysis would be prepared for 33 units of local and regional public administration by the middle of the year 2002. This output has been produced. Besides, 33 territorial self-government units adopted their institutional development plans and launched the implementation by the middle of 2002. This indicator has also been achieved. Indicator 31/06/2003 30/09/2003 31/12/2003 31/03/2004 30/06/2004 30/09/2004 Difference 30/09/2004 Quality of management in territorial self-government units Expected outcome Table 51. % of territorial self-government units participating in IDP, which improved the quality of management – until 12/2003 (%) 60 - 84 84 84 84 94 34 % of territorial self-government units participating in IDP, which improved the quality of services – until 12/2003 ** (%) 30 - 32 32 32 32 94 64 * Institutional analysis repeated and results compared with those from the beginning of the program ** Institutional analysis repeated and results compared with those from the beginning of the program Source: Project Management quarterly reports, Project documents Figure 47. Quality of management in territorial self-government units (TSU) 100 90 80 70 60 50 40 30 20 10 0 30/09/2003 31/12/2003 31/03/2004 30/06/2004 30/09/2004 % of TSU participating in IDP, which improved management quality % of TSU participating in IDP, which improved services quality Source: Project Management quarterly reports Besides, manuals recommending the Institutional Development Program were issued: • “Institutional Development. A Manual for Territorial Self-Governments” • “Institutional Analysis of Gmina Administration Office. A Guide for Self-governments”, Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 243 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka • “Institutional Analysis of Starosta Office. A Guide for Self-governments”, • “A catalogue of Institutional Development Tools. A Guide for Local Selfgovernments”, • “Management in Territorial Self-Government. Best Practices”, • “Ethical Aspects of Territorial Self-Government Activity. A Guide for Territorial SelfGovernments”, • “Ethical Aspects of Territorial Self-Government Activity. Exercise Books”, • “Project Management. A Guide for Territorial Self-Governments”. These manuals were sent to all territorial self-government units in Poland. They are also available on MIAPA’s web site. The evaluation survey has shown that they are known in those territorial self-government units that participated in the ID pilot program and not known in the territorial self-government units that participated in the dissemination program. Even if they can be found in the office, they are not used by the selfgovernment authorities and administration. Moreover, manuals promoting Institutional Development and addressing particular levels of self-government administration were issued. The evaluation survey has shown that they are not known to territorial self-government units’ representatives. The remaining outputs were prepared in the electronic version only and they can be downloaded from MIAPA’s web site (www.mswia.gov.pl/pri): • “Institutional Analysis” for Marszalek Office, • “Institutional Analysis” for Voivodship Administration Office, • “Computer Application Facilitating Self-Assessment of Gmina Administration Office Institutional Development Status”, • “Appendices to the Catalogue of Institutional Development Tools”, • “Quantitative Indicators of Public Services Performance”, • “Quantitative Indicators for Management Areas”, Outcomes and outputs of the All-Poland Training Program Specific data of the Training Program are presented in the table below. 244 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 52. Task Assumed and achieved outcomes of the Training Program Expected outcome Achieved outcome 31/03/2005 60 47 2. Number of persons Assumed participation of 7 trained representatives from each STU on the average 420 367 3. Number of training Number of training manman-days days provided 7.000 5.430 Number of TSU which reported the intention to attend training and were selected by VPIU 540 636 Training about 4200 representatives of 540 TSU in 9 management areas 540 595* 3. Number of training Number of training manman-days days provided 63.000 71.411 4. Number of Total number of projects management projects prepared in the pilot phase prepared and the main phase 540 589** 5. Number of projects the implementation of which was commenced Commencement of implementation of 67% of the management projects prepared 360 432 (73%) 6. Number of TSU Measured by the number of participating actively visits to the web site and the in “distance learning” number of e-mail contacts sessions" 300 629 Indicator Definition Pilot training for 1. Number of TSU Training TSU participating in regions training Training TSU 1. Number of TSU representatives selected to attend the in 9 training management areas 2. Number of TSU attending the training STU for two * The number of persons trained amounted to 4 195. **47 in the pilot phase and 542 in the main phase. Source: Reports of the MIAPA Project Team, MIS monthly reports Outcomes and outputs of the Dissemination Program Depending on the type of initiative, from 72 units (the Competition), through 823 (workshops on ethics), to 1225 units (IDP regional conferences) participated in the Program. 1225 territorial self-government units whose representatives attended conferences where IDP method was disseminated, acquired relevant knowledge and have been prepared to analyse their own institutional potential and the management status on the basis of IDP method and using the tools developed under IDP and consequently – they are ready to implement improvements in management. The remaining 1567 self-government units that did not attend the conference, were sent printed materials and CDs with software. 1969 units that did not accept the invitation, were sent manuals and training materials. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 245 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 53. Assumed and achieved outcomes of the Dissemination Program Indicator Explanation Number of TSU not participating the Program, whose representatives participated in the Dissemination Program At least one representative of each TSU not participating in RDP participated in any form of Program experience dissemination Number of manuals distributed among TSU which did not participate in the Program Preparing an institutional development analysis manual, preparing and implementing institutional development plans with the use of efficiency indicators Expected outcomes Achieved outcomes 30/09/2004 Difference 30/09/2004 600 1225* 625 2800 2800 0 * Additionally, 830 workshops on ethics were organised. They were attended by TSU representatives. The assumed outcomes apply to the number of Territorial Self-government Units which did not participate in IDP, but whose representatives participated in the dissemination program. Source: Project documents, Project Management Quarterly Report no 16 Outcomes and outputs of the MIAPA Support Program The process of building the potential of local and regional administration was expected to be supported by the system disseminating the experience and the implementation methods of Institutional Development. Two indicators were envisaged for this task: • The number of training organisations offering courses in TSU management. The expected outcome was to show 3 such organisations. • The number of consulting companies offering TSU assistance in improving their management systems with the use of tools developed under the program – it was assumed that 7 such tools should be developed. On the other hand, the system aiding management quality improvement was expected to be supported by the means of developing recommendations of legislative changes. Table 54. Expected and achieved outcomes of the MIAPA Support Program Indicator Expected outcomes 30/09/2004 Difference (30/09/2004) Percentage of recommendations of changes in legislation implemented by legal acts (%) 10 16 6 Percentage of recommendations of changes in legislation introduced to draft laws and legal acts (%) 25 31 6 Source: Project documents, Project Management Quarterly Report no 16 It was assumed in both cases that at least 2 recommendations would be prepared for each of 9 management areas. 246 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka It was also planned to provide support in elaboration of – inter alia – two draft agreements of government and self-government institutions concerning creation of a system of improving the management quality in territorial self-government units. Table 55. Outcomes expected and achieved in MIAPA Support Program – agreement of government and self-government institutions concerning creation of a system of improving the management quality in TSU Indicator Explanation An agreement in writing between selfgovernment organisations and a competent government agency (agencies) exists and is implemented. The agreement provides for the rules of co-operation aimed at improvement of the management quality in TSU. 2 projects are implemented at least. Expected outcomes 30/09/2004 Difference (30/09/2004) 2 3 1 Source: Project documents, Project Management Quarterly Report no 16 The following outputs intended for the Administration Department of the Ministry of Interior and Public Administration (MIAPA) were produced under MIAPA Support Program: • Introduction to the Institutional Development Program; • Measuring the Quantity and the Quality of Public Services as an Element of the Institutional Development Program; • Up-dated Tables of Competence for Gminas, Powiats and Voivodship SelfGovernments. These three studies were made available to self-governments via MIAPA web site (www.mswia.gov.pl/pri) and were discussed during the conferences organised under the dissemination program. Moreover, analyses were prepared for MIAPA to identify improvement opportunities in the field of legal solutions ailed at improvement of the efficiency and the quality of management in self-governments and in government administration in voivodships. The effects of the Team’s work constituted a basis for MIAPA to analyse the existing condition of the legal environment in which selfgovernments functioned and a source of recommendations for MIAPA concerning the desirable directions of legislative changes, including proposed amendments to concrete legal acts leading to elimination of barriers to bottom-up improvement of management efficiency and quality in self-governments. The following studies were prepared within the frames of this task: • Legal Bases of Public Administration Functioning in the Polish Republic; • Diagnosis of the Territorial Public Administration Condition in Poland; • Identification of Legal Barriers to Efficient and Effective Functioning of Local and Regional Public Administration. It is difficult to determine, what were their immediate effects, as in the recent years MIAPA did not report any legislative initiatives that would be related to or result from the Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 247 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka studies listed above, especially those presented in document “Identification of Legal Barriers to Efficient and Effective Functioning of Local and Regional Public Administration”. 4.3 Summary 4.3.1 Sub-Component B3 Cost-Benefit Analysis When evaluating the economic effect of Sub-Component B3, one should compare it with other activities of this type performed by the public administration. Some of the activities, such as IDP, were unique, so that it is difficult or even impossible to make any comparisons. It is also difficult to compute the efficiency of IDP with costs of dissemination program. There are no data enabling one to state explicitly, how many projects were implemented after the dissemination program. When talking about the efficiency of the All-Poland Training Program measured by the number of projects that were prepared – the average cost of elaborating one project amounted to EUR 4846. The cost of one training per one TSU participating in the program was EUR 4462, while the cost of training per one trainee was EUR 628. It is difficult to evaluate the efficiency of the training program, as there are no data comparable with similar undertakings in Poland. In the future, it will be possible to assess, to what degree the knowledge TSU acquired enables them to prepare projects on their own, without the need to use external consulting companies. A half of the interviewed participants of the training program identify additional benefits for TSU resulting from training. In their opinion, this is increased investment attractiveness of their gmina and growth of gmina’s income first of all. Some respondents indicated also increased personal income and development of tourism. 248 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 56. Utilisation of WB loan funds for implementation of Sub-Component B3 tasks in the years 2000-2004 (PLN) It. Task Utilisation of WB loan funds 2000 2001 2002 2003 2004 PLN total 1 Institutional Development Program 0.00 461 327.72 5 555 798.69 5 972 608.77 2 014 934.26 14 004 669.44 2 Training Program 0.00 1 191 600.00 2 105 634.20 6 619 582.40 1 257 311.89 11 174 128.49 3 Project Co-ordinator 0.00 204 137.85 240 038.97 226 861.63 128 554.79 799 593.24 4 Institutional Development Specialist 0.00 74 464.66 122 831.44 163 357.24 69 955.48 430 608.82 5 Training Specialist 0.00 48 433.14 142 012.53 163 220.10 34 605.41 388 271.18 6 Monitoring and Evaluation Specialist 0.00 24 717.09 119 445.44 89 675.19 64 791.89 298 629.61 7 Short-term Consultants 0.00 47 871.22 0.00 0.00 18 100.00 65 971.22 8 Reference Team 0.00 243 448.32 91 134.73 521 694.63 1 013 102.57 1 869 380.25 9 Evaluation Study 0.00 0.00 0.00 0.00 0.00 0.00 10 Promotion and dissemination 0.00 0.00 0.00 0.00 1 194 996.58 1 194 996.58 11 IT equipment 0.00 0.00 0.00 0.00 4 673 558.00 4 673 558.00 0,00 2 296 000.00 8 376 896.00 13 756 999.96 10 469 910.87 34 899 806.83 TOTAL: Source: Data MIAPA Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 249 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 57. Utilisation of budget funds for co-financing Sub-Component B3 tasks in the years 2000-2004 (PLN) It. Task Task budget Utilisation of budget co-financing funds 2000 2001 2002 2003 PLN total % total 1 Institutional Development Program 3 090 000 0.00 2 Training Program 2 415 000 0.00 10 291.58 3 Project Co-ordinator 176 000 0.00 44 924.51 52 158.71 48 966.79 27 912.35 173 962.36 98.84 % 4 Institutional Development Specialist 92 000 0.00 16 382.23 26 312.71 34 886.88 15 147.02 92 728.84 100.79% 5 Training Specialist 84 000 0.00 10 655.29 32 001.99 33 472.91 7 593.86 83 724.05 99.67% 6 Monitoring and Evaluation Specialist 67 000 0.00 5 437.76 25 869.86 19 668.27 13 835.07 64 810.96 96.73% 7 Short-term Consultants 13 000 0.00 9 463.38 0.00 0.00 1 900.00 11 363.38 87.41% 8 Reference Team 0 0.00 0.00 0.00 0.00 0.00 0.00 - 9 Evaluation Study 0 0.00 0.00 0.00 0.00 0.00 0.00 - 10 Promotion and dissemination 370 000 0.00 0.00 0.00 0.00 262 486.25 262 486.25 70.94% 11 IT equipment 1 053 000 0.00 0.00 0.00 0.00 1 168 388.82 1 168 388.82 110.96% 7 360 000 0,00 202 000.00 1 833 959.00 3 105 703.99 2 218 147.20 7 359 810.19 100.00% TOTAL: 104 845.25 1 050 685.60 1 490 907.03 2004 646 930.13 1 477 802.11 Source: Data MIAPA 250 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 444 275.22 3 090 713.10 100.02% 276 608.61 2 411 632.43 99.86% Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka 4.3.2 Long-Term Impact of Sub-Component B3 The impact of Sub-Component B3 should be evaluated in a few aspects. Identification of program activities by territorial self-government units (TSU) participating in it is the first aspect. Activities relating to dissemination of the program effects produced during the program is another aspect. Dissemination and utilisation of achievements after the program completion is the third aspect. An analysis of interviews with participants of IDP and Training Program one may state that the respondents did not entirely identified themselves with one of the main objectives of RDP, i.e. increasing the non-farm employment in rural areas. The largest number of participants identified themselves with the objective relating to self-governments’ adaptation to functioning in EU understood as the improvement of infrastructure functioning and of ability to use structural funds – sometimes defined as “the skill of writing applications”. Few TSU identified themselves with the objective the Project Team had defined as improvement of the quality of TSU functioning. This thesis is supported by the fact that only 5% of applications covered changes in self-government administration directly and dealt with the implementation of the “New Public Management” rules. The analysis of training programme participants’ questionnaires and of interviews with IDP participants shows that most of them approached to particular Components of the Project as separate, individual “World Bank Programs”. The evidence can be found in respondents’ answers, where participants of the training program indicated their participation in IDP – in questionnaires prepared for participants of the All-Poland Training Program, 52% of them stated that they participated in the Institutional Development Program concurrently. At the same time, unfamiliarity with the Project is confirmed by answers about the number of team members. It was assumed that each team should consist of 6 to 7 employees of administration and gmina council. Only 35% of respondents quoted this number, the rest quoting from 1 to 8 or even more. In many cases, RDP was identified with Components B2 or C first of all. When evaluating the sustainability of outcomes, one should consider them from two points of view: TSU participating in activities of Sub-Component B3 – IDP and Training Program and the institution implementing the Sub-Component – MIAPA. In some analysed cases, TSU managers decided to implement further innovations proposed within the frames of IDP. The others do not continue transformations, as they are inadequate to their needs in their opinion. One should state however that the process of organisational culture transformation has been commenced and that its effects may strengthen in the future. Especially that effects of IDP can be observed at the regional level even now, e.g. in Małopolska the program provided significant assistance in putting the finishing touches on the conception and the construction of “Wrota Małopolski” www portal and indirectly – in developing the “digital office” module. On the other hand, in the opinion of the Training Program participants – representatives of TSU, projects developed under these programs did not contribute to any improvement of co-operation between TSU and the private sector. A co-operation like this was supposed – such the RDP authors’ intention had been – to result in creation of new jobs in rural areas. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 251 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Table 58. Outcomes of projects in Sub-Component B3 Did the project implementation contribute to improvement of co-operation between TSU and the local private sector? Proportion Yes 22.6 No 32.3 Do not know, hard to say 45.2 Total 100.0 Source: PBS survey, March 2005 On the other hand however, entrepreneurs from gminas covered by IDP identified the improved level of services in their gminas. But the high sustainability of the Training Program impact manifests itself through the fact that more than a half (57.4%) of TSU participating in the All-Poland Training Program submitted also other projects, apart from those worked out under the Training Program. Every fourth gmina plans to do this in the nearest time. Similarly as in case of projects developed under the Training Program, SAPARD and ZOPRR were the most popular sources of financing here and sometimes - Phare and SPO ROL. Most of the projects are being implemented now. It is definitely worth stressing that in the opinion of the vast majority of respondents, the participation of institutions representatives in the Training Program was very significant for writing the projects. 16.1% of the respondents state that participation in the Program was very important and that they would not write their new projects without it. 74.2% are of the opinion that although they would write the projects without participation in training, participation in the Program made it much easier for them. Only two out of 31 respondents, when answering the question about the effect of the Program on new projects writing, stated that they would write their project anyway, irrespective of participation in the Program. Activities undertaken by MIAPA – responsible for Sub-Component B3 implementation – represent the second level at which the sustainability of Program effects can be assessed. The dissemination program implemented by MIAPA can be divided into two phases. The first of them was executed within the frames of Sub-Component B3 by the Ministry staff and by the Project Team, the second – after the co-operation with the team was closed. To assess the first phase of dissemination, one should stress that the Role of MIAPA was noticeable. The Dissemination Program had been inspired by MIAPA and implemented in co-operation with four organisations – representatives of TSUs: the Polish Union of Rural Gminas, the Union of Polish Cities, ZPP, UMP. Representatives of MIAPA participated in all 16 regional conferences and in the “Self-government Management Leader” competition as members of the competition commission and in the group of experts. Publication of all outputs produced during implementation of Sub-Component B3 on www sites was one of the elements of dissemination. The second phase of dissemination took place after activities of Sub-Component B3 were completed, i.e. from January 2004 on. When this phase was evaluated, activities related to further dissemination of Sub-Component B3 achievements were followed. 252 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka When analysing activities half a year after Project completion and almost one year after dissemination program completion, the evaluator has found out that MIAPA www sites, in their part informing about the Institutional Development Program, still include information about activities that had already taken place – an invitation for a conference dedicated to the application of Institutional Development methods, an invitation for training workshops dedicated to ethics and information about the “Self-government Management Leader” competition – all of them announced as current events. At the same time, in case of selfgovernment organisations’ www sites, information about the outputs of Sub-Component B3 is absent or obsolete. The tools produced by Sub-Component B3 did not fall in the areas of interest of such self-government press titles as Wspólnota or Gazeta Samorządu i Administracji. Besides, the next “Self-government Management Leader” competition has not been announced so far. Thus, one should believe that neither the Government, nor the self-governments have not undertaken any activities after Sub-Component B3 completion in order to continue dissemination of the Program with the improved and more responsible local management in view. It seems that to ensure the sustainability of Sub-Component B3 outcomes, MIAPA should play a more significant role – similarly as in the Dissemination Program. For example, MIAPA in co-operation with self-government organisations might continue the “Selfgovernment Management Leader” competition. This might be done, using similar rules as are used in organisation of the “Safe Gmina” competition. In this case, MIAPA mobilises gminas, to release initiative conducive to co-operation and improvement of forms of ensuring inhabitants’ piece and safety in large and small cities and villages. To strengthen the process of dissemination, it is worth keeping organisations that represent self-governments involved. But, following the dissemination program experience, it is worth creating a Government program of support for institutional transformations in administration. At the same time, when creating a program like this, one should provide the required funds, as any activities related to making the selfgovernment administration more professional require external support. 4.3.3 Activities of Sub-Component B3 Sub-Component B3 was divided into the following basic parts: • Institutional Development Program (IDP), Zachodniopomorskie and Małopolskie voivodships; • Training program implementation (all voivodships); • Dissemination Program (all voivodships). which covered Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Podlaskie, 253 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Figure 48. Schedule of Sub-Component B3 Source: Consultant’s own analyses based on Project Management Interim Reports, PCU 254 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka Implementation of the Institutional Development Program Selecting 33 pilot units to the Institutional Development Program (IDP) was the first step in implementation of Sub-Component B3 (the process of selecting voivodships commenced in March 2000, while powiats and gminas – in 2001). By the end of June 2001, 75% of all Institutional Development Plans were approved. The program participation contracts were signed in July 2001. At the same time, a contract for technical assistance was signed between MIAPA and the Institute of Market Economy Studies (IBnGR). The assistance covered preparation of legal analyses and assessment of Program outputs. In September, contracts were signed with IDP implementers (Małopolska Szkoła Administracji Publicznej / Canadian Urban Institute) and Fundacja Rozwoju Demokracji Lokalnej (the Foundation for Local Democracy Development) contracted to run the Training Program. Moreover, the technical assistance included preparation of the first report titled “Legal Bases of Public Administration Functioning in the Polish Republic”. IDP implementers began their work in October 2001 with preparing a schedule, a draft “Best Practice” catalogue, Program methodology and efficiency indicators. Opening seminars (November 2001) and pilot institutional analyses (December 2001) were the next steps. The pilot program had to be postponed in voivodship offices due to the Parliament election and the resultant changes in voivodship administration. Outcomes of pilot activities performed in the remaining units were presented during a seminar organised at the beginning of the year 2002. By the beginning of July 2002 institutional analyses and analyses of equipment-related needs were carried out in 33 units. The task included institutional analysis, preparation and implementation of Institutional Development Plans by Territorial Self-government Units (TSU). Reports of MIAPA Project Team were the source of this information. In the second quarter of 2002, the Canadian Urban Institute and Małopolska Szkoła Administracji Publicznej prepared institutional development plans where proposed specifications of equipment to be purchased and detailed plans of training were presented. By the end of 2002, a list of required computer equipment to be purchased under IDP was drawn up. The implementation of IDP commenced in January 2003 and the Program was closed in December 2003. A bidding procedure for the computer equipment procurement was settled at the end of 2003 and deliveries were completed by February 2004. At the same time, a decision was made to launch the program disseminating achievements of SubComponent B3 and a respective annex to the loan contract was signed. The Program was implemented in the year 2004 in the form of conferences, training and workshops on ethics. 30 TSU and 3 voivodship administration offices participated in the Institutional Development Program. Implementation of the All-Poland Training Program On 21 September 2001, MIAPA signed a Training Program contract with the Foundation for Local Democracy Development. The contract provided for training 600 teams of 6 – 8 members each from selected self-government units. The Training Program Implementation commenced in October 2001. An analysis of training needs was the first stage. In December, the NSC allocated training places to voivodships and the Project Team sent invitations to attend the pilot training. The outcomes were presented at conferences in May 2002. By mid-January, the services provider prepared nine training packages within the frames of the Training Program. The Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 255 Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka packages addressed the issues included in the IDP, the outcomes of the training needs analysis and the conclusions from the first stage of IDP implementation including case studies relevant to the subjects of 5 training programs. Trainers were trained and pilot training commenced in February 2002. The pilot program was completed on 14 May 2002. 51 units from eight powiats joined the Program. 47 units completed it and 370 persons were trained in total. By the end of the year 2002 a set of training materials was prepared, finishing touches were put on the methodology and participants were recruited. In September 2002, the NSC admitted cities with the population up to 40 thousand to the Program, postponing the closing date of the Program until January 2004 at the same time (the reason: delays resulting from self-government election). In the first quarter 2003, the All-Poland Training Program was launched. In the second quarter 2003 it entered its main phase. The number of candidates was twice as large as the training capacity allowed for. In total, 636 territorial self-government units were recruited. In the third quarter 2003 the All-Poland Training Program was completed in most of the voivodhips. Finally, the Program was attended by 595 TSUs (4 195 trainees). The task was completed in the year 2003. 4.3.4 Conclusions and Recommendations for the Future To sum it up, one may state that RDP was necessary in the spectrum of Sub-Component B3 in the period when it was being implemented, as it prepared the Polish selfgovernment for joining EU structures. The Project objectives complied with the State policy objectives outlined in such strategic documents as the National Strategy for Regional Development, the National Strategy for Employment Growth and Human Resources Development and the Cohesive Structural Policy for Agriculture and Rural Areas Development. Assessing the effectiveness of Sub-Component B3 in general, one should state that it was very high at the level of units implementing IDP and All-Poland Training Program, especially in the area of activities related to absorption of pre-accession and structural funds. Nevertheless, the efforts aimed at implementation of tools proposed within B3 to transform the model of TSU management considerably were not as successful. The bureaucratic model of management still remains very common. One should expect however, that the changes that have been initiated will produce the expected result in the course of time, providing that MIAPA and self-government organisations show more commitment. As far as the innovative nature of Sub-Component B3 is concerned, one should pay attention to the features conducive to dissemination of the solutions proposed within its frames. The capability of co-ordinated expansion should be one of important qualities of innovativeness. This capability will be ensured by the effective implementation of legal regulations. One should remember, that implementation of innovations in the selfgovernment administration does not result from TSUs’ need of innovativeness, but it is rather determined by the legislation requirement. In the future, legal frames for implementing innovative programs related to the “New Public Management” should be created first of all. Only then it will be possible to implement the programs of transformation efficiently and effectively. One should consider in the future, to what degree national organisations representing the self-government are 256 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Sub-Component B3 Beata Ciężka able to perform tasks related to implementation of modern management methods on their own, and to what degree this should be the responsibility of the State. In order to carry on the sustainability of the Project impact, one should envisage funds and organisational conditions for further dissemination of achievements. It is impossible to evaluate the effectiveness of activities undertaken within the frames of B3 and aimed at achievement of one of RDP objectives – increasing non-farm employment in rural areas. The basic difficulty results from the fact that Sub-Component B3 was implemented separately from other Components and Sub-Components of RDP and, as the Project Team Manager said, “by the way”. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 257 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) 5 Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 259 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) 5.1 Top-Down Analysis 5.1.1 Identification of the problems which Component C was supposed to help solve The objective of Component C consisted in improving and modernizing infrastructure in rural areas in order to encourage investment by the private sector, facilitate the creation of non-agricultural employment and improving the state of the environment and the health of the population. The objective of this Component was also to create alternative organizational and institutional structures with a view to ensuring the delivery of sustainable and accessible municipal services and introducing a market oriented approach to the process of building and operating infrastructure. The area of the Component covered all 16 voivodships. Component C contributes to the achievement of two of the three general Project objectives: increasing non-agricultural employment in rural areas and supporting the process of decentralization of local government and regional development. According to the Project Appraisal Document, to which Component C was supposed to respond, the level of development of rural infrastructure was very low – both from the point of view of living and working conditions and the possibilities with regard to developing extra-agricultural areas of activity. The significance of infrastructure in socio-economic development Technical infrastructure defined as a “set of basic facilities and institutions that are necessary to ensure the proper running of the economy and the organization of life in a particular area” constitutes a material resource base that is integrally linked with a specific area and, which serves both production and consumption purposes119. The significance and role of technical infrastructure in socio-economic development derives from its very essence and thus from the fact that it forms the foundation and basis for any economic activity and is the principal determinant for people’s living conditions in the broadest sense of the term. The level of infrastructure development determines the investment attractiveness of a given area both to local and external investors and therefore creates the conditions for the multi-functional development of rural areas. Areas with comprehensive infrastructure provision have a greater probability of attracting investors, whilst areas that are neglected in this respect burden potential investors that wish to set up and run their own company with additional costs associated with building roads, water supply systems and wastewater disposal facilities, central heating systems, and telecommunications infrastructure.120 There is general agreement with regard to the significance of infrastructure in socioeconomic development The view is held that building infrastructure facilities is a necessary condition – although not always one that is sufficient on its own – for creating the technical and economic basis for the location of means of production for the purposes of direct manufacturing as well for creating the basis for developing varied social functions121. It has been empirically proven that investments in infrastructure can 119 A. Ginsbert-Gebert, C. Rudzka-Lorentz, M. Woźniak, A. Zielińska-Kołtyś, Infrastruktura komunalna a funkcjonowanie systemu osadniczego, (in: Gospodarka przestrzenna, Wrocław 1984) 120 PAD, p. 14. 121 K. Jarosiński, Finansowanie inwestycji komunalnych w Polsce w warunkach samorządności lokalnej. SGH, Warszawa 2003. 260 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) stimulate economic growth. Technical infrastructure has an impact on manufacturing costs (the cost effectiveness of production), the speed of the diffusion of innovations and the opportunity to access financial markets. Infrastructure also improves how markets function, makes it easier to flexibly adapt to changing conditions122 and impacts on the diversification and growth of farmers’ incomes. The scale of entrepreneurship is clearly correlated with the economic standards of rural areas and in particular with the development level of infrastructure123. The extent to which infrastructure development correlated with the level of economic development at a gmina [commune] level at the starting point of the RDP Rural Development Project is indicated by the results of the statistical analyses below concerning the scale of unemployment and the employment structure in groups of gminas with extremely unfavorable, average and the highest development level of technical infrastructure124. Table 59. Economic situation in groups of gminas with varied infrastructure levels Indicators describing: Groups of gminas with the following level of infrastructure: Lowest Medium Lowest Open unemployment rate 9.2 13.8 16.2 Those working outside the individual agriculture sector as compared with the population of productive age. 27.2 12.9 9.2 Source: Own calculations based on indicators elaborated by A. Rosner, Wiejskie obszary kumulacji barier rozwojowych, IRWiR PAN Warszawa 2002 The data compiled thus confirm the existence of a statistical relationship between the level of infrastructure provision in rural areas and their economic state of affairs, which in this case was indicated by the indicator of unemployment and the percentage of those working outside individual agriculture in relation to the population of productive age. The highest level of infrastructure was applicable to gminas with a higher percentage of population employed in commerce and services than the average for the Polish countryside and to gminas with developed tourism and recreation functions. In contrast, the gminas with the lowest relative level of economic infrastructure were monofunctional gminas with agriculture being a dominant function and which are problematic from the point of view of demographic processes and structures (these are characterized by a high aging rate of the rural population). The analyses of differentiation of rural areas in various spheres prove that there is generally a very large spatial coincidence of where areas with a favorable infrastructure situation are located with areas with a relatively high level of economic activity. 122 J. Kulawik, Problemy finansowania infrastruktury ekonomicznej wsi i rolnictwa, IERiGŻ, Warszawa 1999. 123 B. Wojciechowska-Ratajczak, Podstawowe uwarunkowania ekonomiczne oraz kierunki polityki strukturalnej wobec wsi i rolnictwa w Polsce, IRWiR PAN, Warszawa 1999. 124 A typology of gminas from the point of view of infrastructure has been carried out on the basis of 7 basic indicators that characterized the following: the percentage of homes in a gmina that are connected up to the water supply network, the percentage of homes in a gmina that are connected up to the sewage network, the percentage of the population in the gmina served by sewage works, the number of telephone subscribers per 1000 persons, the length of extra-urban gmina roads with 2 2 an improved, paved surface per 100 km , the number of petrol stations per 100 km and the 2 number of shops per 100 km . More information on the methodology used can be found in: B. Pięcek, Wiejskie obszary kumulacji barier rozwojowych, IRWiR PAN, Warszawa, 2002. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 261 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The scale, spatial differentiation and causes of the problems, to which Component C responds The problem of rural infrastructure; that is its relatively low level of development is the result of cumulative neglect in this sphere throughout the post-war period. The scale of the neglect in this respect is indicated by the fact that at the very beginning of the 1990’s only one in three households (and only 2.3% of agricultural households) was connected up to the water supply system. With respect to the sewage system, this indicator was several times lower and amounted to 1.6% (this amounted to only 0.8% for farming households). In 1992 only 130 municipal solid waste disposal sites were operating in rural areas. In 1990, 41.5% of the population of Poland was served by sewage works; in 1999 the figure came to 51.5%, whereas in 1990 in countries such as Luxemburg, Switzerland, Sweden, Great Britain, Holland, Denmark and Germany this ratio exceeded 90%. In 1996 sewage treatment plants served only 4% of the rural population (in towns this came to 67% - according to the OECD the indicator of the level of sewage works provision for the population in Poland is still one of the lowest in Europe). The gap that separates us from other European countries with regard to the infrastructure provision of rural areas is very large. As early as the end of the 1980’s the percentage of rural households in France that were connected up to the water supply network amounted to 98%, in Denmark this exceeded 90% and at this time all Bavarian villages with more than 500 residents and high housing density had been connected up to the sewage system 125. In Western Europe sewage works serve more than 90% of the population of these countries according to statistical data126. The level of infrastructural neglect in rural areas at the end of the 1980 / beginning of the 1990’s may be demonstrated by the fact that if the rate of development of the water supply system had remained at the same level in the 1990’s (and subsequent years) as in the previous decade, it would take at least 40 years to expand the water supply system to fully meet the needs of rural areas and in the case of the sewage system this could take as long as a century. Despite the acceleration of the development of rural infrastructure in the 1990’s, the concentration of forces and financial resources (both domestic and foreign, co-financing using government and local funds), the commitment of local government and communities, the process of building rural infrastructure at the beginning of the 2000’s had not reached a satisfactory level. In 1999, 60% of rural homes and 72% of farming households were connected up to the water supply system (BDR 1999). Prior to the commencement of the Project 73% of rural localities (village councils “sołectwo”) in Poland had a water supply system, although rural localities that had full provision of this infrastructure facility constituted 64% of all village districts in the country (thus 27% of villages did not have any water supply system at all or were in the process of setting one up). The highest share of villages (exceeding 90%) (fully or partially) equipped with a water supply system was characteristic for the Wielkopolskie, Śląskie and Opolskie voivodships. The lowest level was applicable to the voivodships of Zachodniopomorskie, Mazowieckie, Podkarpackie, Lubelskie, Lubuskie, Podlaskie and Świętokrzyskie. The gap in this sphere between voivodships at opposite ends of the scale in the aspect under discussion is enormous, as it amounts to 42 percentage points. The degree of imbalance 125 Raport o stanie infrastruktury technicznej wsi w Polsce na koniec 1994 roku. MRIGŻ, Warszawa 1995. 126 Rocznik Statystyczny Rzeczypospolitej Polskiej, GUS, Warszawa 1999. 262 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) between individual gminas with respect to the development of water supply networks is even greater, as it falls within the range of 0 to 99% and therefore has reached a maximum extent of variance. Table 60. Percentage of rural localities that have a communal water supply and sewage system broken down by voivodship Voivodship: Percentage of rural localities that Percentage of rural localities that have a communal water supply have a communal sewage system system Full or partial provision Of which, full provision Full or partial provision Of which, full provision Dolnośląskie 78.5 96.9 11.3 28.9 Kujawsko-pomorskie 87.3 86.7 7.8 57.4 Lubelskie 67.4 95.6 4.1 49.7 Lubuskie 70.3 90.0 12.8 48.1 Łódzkie 89.2 89.7 2.3 51.4 Małopolskie 65.0 79.0 10.9 77.9 Mazowieckie 58.7 91.9 2.5 65.8 Opolskie 91.6 94.3 4.2 95.5 Podkarpackie 59.7 76.2 16.9 50.4 Podlaskie 70.4 81.4 2.9 44.8 Pomorskie 82.7 84.4 15.6 66.4 Śląskie 93.9 90.0 10.0 64.1 Świętokrzyskie 71.0 95.6 4.4 57.4 Warmińsko-mazurskie 69.1 92.0 8.0 52.1 Wielkopolskie 94.1 92.9 7.1 52.4 Zachodniopomorskie 51.9 89.8 10.2 46.6 TOTAL 72.9 83.7 6.6 22.8 *number of connections per 100 residents Source: Own calculations based on MRIRW data, 2000, Warszawa The share of rural dwellings connected up to the sewage system was incomparably lower than those linked up to the water supply system and amounted to 4.5% (BDR 1999), for agricultural households the analogous indicator was slightly higher and was around the level of 7.7%. For more than half of the gminas in the country the percentage of homes connected up to the sewage system did not exceed 3.5% on average and for only 2.6% the ratio of connections to the sewage system attained a value of more than 30%. Prior to the commencement of the Project, a sewage system was in operation to varying extents in 6% of rural localities, however only not quite 3% of villages in Poland had been connected up to the sewage system in their entirety. The following voivodships had the most favorable states of affairs in this respect: Podkarpackie (the share of dwellings connected up to the sewage network was close to 17%) as well as the Pomorskie and Lubuskie voivodships. In contrast, the Łódzkie and Podlaskie voivodships were in the least favorable position. There was a sevenfold difference as regards the proportion of homes linked up to the sewage system between the Podkarpackie and Łódzkie voivodships, which were at opposite ends of the scale with respect to the level of sewage infrastructure. At the gmina level the extent of the variance in this respect fell in the range Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 263 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) of 0 to 69% and was therefore also very large. At the voivodship level, the imbalance between the level of development of the water supply and sewage system was very considerable and it was generally greatest where there was the highest ratio of connections to the water supply system. In 1999 the length of the sewage network was six times shorter than the water supply system and for every one sewage connection there were nearly 11 water supply connections (more than 18 in 1996). Despite the fact that the relationship between the number of sewage and water supply connections is improving, the absolute difference is growing, which means that the problem of water and sewage management in rural areas is deepening. The quantity of water (in dam3) supplied to rural households was 5.5 times higher than the quantity of collected wastewater (BDR 1999). Table 61. The imbalance between the development of the water supply and sewage system in 1999 Indicators Quantity Number of water supply connections (in thousands of units) Number of sewage connections (in thousands of units) The difference between the number of water supply and sewage connections 3 Quantity of water supplied to households (in dam per annum) Quantity of wastewater collected (in dam3 per annum) The difference between the quantity of water supplied and the quantity of wastewater collected (in dam3 per annum). 2 205.7 209.7 1996.0 319 074.0 73 897.8 245 176.2 Source: BDR data 1999, GUS, Warszawa The imbalances between the development of the water supply and sewage network (two systems that in the nature of things are closely functionally linked with each other) are a very unfavorable phenomenon in many ways. When households are connected up to the water supply system water consumption rises, which entails a proportional increase in sewage and as a consequence a rise in the extent of pollution of water and of the entire environment, which also has an impact on living conditions (including the state of health) of the rural population. The lack of a sewage disposal network is associated with worse investment conditions (the lack of comprehensive facilities). This problem is not solved by local facilities and farm sewage treatment facilities, the total number of which amounted to 34.3 thousand in 2000 (these included farms of more than 1 ha that numbered only 758 and therefore only not quite 4% of households had them)127. Very considerable neglect was noted in rural areas in Poland with regard to building gmina roads (company roads and access roads to farming land) and the organization of waste landfill sites. Storage in landfills is the common method of disposing of solid waste in rural areas. Many communities lack landfills or have facilities that do not fulfill legal, safety and sanitary requirements (PAD page 8). This is confirmed by statistical data, which indicate that in 1999 there were 1130 units of landfills (with a total area of 2083 ha). Generally the overall density of roads serving agriculture was sufficient in the majority of voivodships. However, their distribution necessitated large changes in spatial configurations (bringing them in line with the transformation of the structure of the 127 264 MRIRW data, Warszawa 2000. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) countryside and the area development plan for a given area) and their technical condition also needed to be improved. In 1997 the Ministry for Agriculture and Food [MRIGŻ] anticipated that municipal paved roads had to expand by approximately 50 thousand km (by 2010) in order to improve the situation with regard to roads128. A review of the condition of roads in 112 gminas performed by NIK in 1997 indicated that solely a 1/4 of the length of gmina roads was fit for use without restrictions stemming from their poor technical condition. Around 80% of the length of gmina roads needed to have their surfaces, pavements and drainage ditches repaired. In every four gminas permanent restrictions to use (restrictions to speed and to the weight of vehicles) had been introduced that hampered traffic flow, due to the poor technical condition of roads. Furthermore, in half of the examined rural gminas some roads were not periodically passable in spring and autumn periods due to rainfall. The poor technical condition of roads also had an impact on the finances of gminas. One in every five of them granted and paid out compensation for damage to vehicles or to health caused by the poor state of the roads. The poor technical condition of roads is one the weaknesses of gminas in Poland most often mentioned in development strategies. The large number of pedestrians involved in accidents resulting from the necessity of having to walk on the roadway is one of the major reasons for the large number of fatal incidents outside built-up areas. The scale of this risk is best illustrated by the fact that nearly one in three accidents involving pedestrians outside built-up areas results in a fatality. In built-up areas this occurs more rarely – one in nine accidents. The role and significance of infrastructure in socio-economic development (which has been empirically confirmed) on the one hand and on the other the enormous neglect in this sphere in rural areas (as well as the very large spatial scope of imbalance between regions and also within their borders) fully justify the assumption adopted in the Project that inadequate development of infrastructure constitutes one of the most important barriers standing in the way of the development of rural areas (PAD page 8). The actual and forecast speed of development of technical infrastructure in the years 1996-2003 In the 1990’s the water supply system was one of the fastest developing elements of technical infrastructure in the countryside. In the period of 1996-1999 the number of connections to the water supply network in rural areas increased by more than 50% and the length of the water supply system grew by 25%. The greatest rate of change pertaining to the development of the water supply system was noted in the following voivodships: Podlaskie, Świętokrzyskie, Małopolskie and Mazowieckie (and thus those where the level of development of this infrastructure element was the lowest). Analysis of the rate of development of the water supply network in the years 1996-2000 and extrapolation of the results of how this rate would be maintained over the duration of the RDP indicated that in the sphere under discussion a fall in the annual average growth of the length of the water supply system had occurred. This was mainly the result of attaining a relatively high level of provision of gminas with this infrastructure element at the point when the project started. Hence, this contributed to higher extrapolated achievements than actual ones with respect to the length of the water supply system in the year 2003 (in this year the majority of the investments realized within the framework of the RDP had been more or less completed, furthermore GUS did not publish data on the above subject for the year 2004). 128 Program rozwoju infrastruktury technicznej wsi do roku 2010, MRIGŻ, Warszawa 1997. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 265 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Figure 49. The extrapolated and actual length of the water supply network in the years 1996-2003 200 000 190 000 180 000 170 000 160 000 k m 150 000 140 000 130 000 120 000 1996 1997 1998 1999 2000 2001 2002 2003 year Extrapolated length of the water supply network in km Actual length of the water supply network in km Source: own work on the basis of the BDR 1996-2003, GUS, Warszawa Focusing activities and resources in the 1990’s on investments in the water supply system (the first and essential element of the development of the water supply and sewage infrastructure) contributed to improving the situation of rural areas in this respect. The level of fulfillment of needs has attained a relatively high degree as it comes to nearly 75%. The overwhelming majority of gminas have completed building of water supply systems. As a consequence, at the end of the 1990’s the level of activity of local government authorities increased in the sphere of expanding the sewage system. The growth projection of the sewage network based on the rate in the period prior to 2000 indicates that the actual rate of change in this respect during the Project was higher than the one forecast on the basis of the rate of change before the Project. As a result, in the years 2000-2003 greater growth of the length of the sewage network was achieved than that which could have been expected from extrapolating the results from previous years. The question, which arises here concerns the role in and significance for the intensification of activities in this sphere of the additional sources of finance for infrastructure investments in the form of the Rural Development Project. 266 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Figure 50. The extrapolated and actual length of the sewage network in the years 1996-2003 35000 30000 25000 km 20000 15000 10000 5000 0 1996 1997 1998 1999 2000 2001 2002 2003 year Extrapolated length of the sewage network in km Actual length of the sewage network in km Source: own work on the basis of the BDR 1996-2003, GUS, Warszawa As we have already mentioned, bringing the rate of development of the sewage system in line with that of the water supply network is an important issue in the sphere of environmental protection and the living conditions of the rural population. Despite the fact that the gap in this respect is increasing, the rate of divergence between the development of these two infrastructure elements showed a considerably lower rate in the years 20002003 compared with the prior period, as indicated by the data included in the table. If in the years 2000-2003 the annual average scale of divergence between the water supply network and sewage system had been maintained the difference would have been 13% higher than the actual one that occurred in 2003. Over the duration of the RDP a change in the tendency took place, which may indicate a systematic improvement in the relationship in this respect. Table 62. Change in the length of the sewage network and the length of the water supply network in the years 1996-2003 Length of network (in thousand km) Year Difference in length of the water supply and sewage networks Water Supply Sewage 1996 123.7 6.5 117.2 1997 136.6 8.5 128.1 1998 146.7 10.7 136.0 1999 154.7 13.3 141.4 2000 161.8 16.2 145.6 2001 167.1 19.0 148.1 2002 173.2 22.9 150.3 2003 179.5 28.8 150.7 (in thousand km) Source: BDR data 1996-2003, GUS, Warszawa Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 267 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) When analyzing the tendencies of changes in the development of the road network, the value of the indicator describing the length of gmina roads with a paved, improved surface was taken into account. In the period of 1996-1999 the length of roads in this category increased by 5 thousand (by more than 18%), whereas in the years 2000-2003 the growth in paved, improved gmina roads was a lot lower, as it amounted to 2.9 thousand km. In this period, the total length increased by only 9%. Analysis of the growth rate of the road network in the years 1996-1999 and extrapolation of the results of how this rate was maintained over the duration of the RDP indicated that in the sphere under discussion a fall in the annual average growth of the length of paved and improved gmina roads had occurred. As indicated by the above graph, we have higher extrapolated results than actual ones with respect to the length of the road network (in the category of gmina roads under discussion) in 2003 at the level of 8.5%. Figure 51. Extrapolated and actual length of the network of paved, improved gmina roads in the years 1996-2003 190 000 180 000 170 000 Extrapolated length of the network of paved, improved gmina roads in km 160 000 150 000 Actual length of the network of paved improved gmina roads in km 140 000 130 000 120 000 110 000 100 000 1 2 3 4 5 6 7 8 Source: own work on the basis of the BDR 1996-2003, GUS, Warszawa The financial aspect of infrastructure development The development of technical infrastructure is a long-term process – both from a technical and economic point of view - and this stems from the specific characteristics of infrastructure facilities and systems, and above all their capital intensiveness. One of the characteristics of infrastructure is its solid form, which also has numerous implications in the economic sphere. It determines whether it is necessary to undertake infrastructure investments on a specific technical scale, which sometimes exceeds the actual requirement that exists for a given service in a particular region (e.g. the gmina). This may lead to a situation where immediately after a facility has been put into operation considerable spare production capacity can arise and this will only be utilized at a later point in time, although it would have already been necessary to incur financial outlays. Another specific characteristic of technical infrastructure is the fact that its development is characterized by a defined sequence of activities stemming from the functionality and the interdependence between individual systems (elements). This means that the solution of 268 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) the issue of, for example water and sewage management has its own sequence of activities, with the completion of each of them facilitating, but also necessitating the undertaking of subsequent actions and only the realization of all the elements (e.g. the water supply network, sewage system, sewage treatment plant) makes it possible to fully meet the needs in this respect. The comprehensive nature of these activities makes it necessary to incur enormous financial expenditure for a long period. The realization of infrastructure investments on a local level is one of the important tasks, for which local government units are responsible. The scale and scope of the investments undertaken is delimited by the amount of local authorities’ revenues, which directly translates into their capacity to finance investment projects using their own funds. In Poland there are great imbalances with regard to economic development and with respect to own income per 1 resident and in absolute terms. The situation in this sphere would seem to indicate a certain relationship - with gminas with low revenues and limited capacity of amassing funds being characterized by relatively poorer infrastructure facilities and relatively larger needs in this sphere. Urban and urban-rural gminas have greater possibilities and a higher level of infrastructure development. Table 63. Gminas’ budget revenues in groups of gminas with diverse infrastructure levels Indicators Groups of gminas with the following level of infrastructure: Highest Medium Lowest Gmina’s budget revenues from their share in state treasury revenues stemming from natural and corporate persons per 1 resident (in PLN) 219.2 135.9 114.4 Gmina’s own revenues per 1 resident (PLN) 813.3 391.7 341.2 Source: Own calculations based on indicators elaborated by A. Rosner, Wiejskie obszary kumulacji barier rozwojowych, IRWiR PAN Warszawa 2002 The data collated in the above table show the tendency of the relationship between the income situation of local authorities and their level of infrastructure provision prior to the implementation of the RDP. In gminas with the relatively highest level of infrastructure development, the level of gmina budget revenue was two times higher and if gmina revenues are expressed per resident then the scope of the imbalance between these types of units increased considerably. Appraisal of the rationale for Component C The majority of Polish gminas has still not attained an adequate standard of local technical infrastructure and the limited capacity of the gminas with respect to financing municipal investments means that the investment requirements exceed the possibilities of financing the investments with current budget revenues in successive years. Local economic factors restrict the capability to undertake such a level of investments that are in line with needs in this respect and which would ensure balanced development of the gmina. Financing of infrastructure investments, particularly large scale ones in situations where limited own funds are available means that the investments need to be co-financed or fully financed with repayable or loan funds. For some local authorities undertaking investments without external financial support is difficult to manage and even impossible Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 269 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) for others (especially for gminas considered to be problem areas from an economic and demographic point of view). In light of the above, it would appear expedient to consider the period, in which it is necessary to make up for the neglect accumulated over many years as an interim one, in which it is indeed through external financial assistance that the development rate of infrastructure is speeded up. Such an approach is justified due to the difficult circumstances of some gminas with respect to their capacity to obtain their own revenues and the growing economic and consequently infrastructure imbalances in our country. At the point of planning and commencing Component C the level of infrastructure in rural areas was still very low compared with needs and the intensification of activities in this sphere in the 1990’s has definitely led to an improvement in infrastructure indicators in rural areas and also to a widening of the spatial differences between individual regions (and also within their borders) as well as between individual gminas and between its individual elements (above all between the water supply and sewage system). The specific characteristics of infrastructure development in rural areas presented herein, particularly its capital intensiveness and the ineffectiveness of local authorities justifies the financial support of infrastructure investments within the framework of the RDP, as it is an important element of the strategy of evening out spatial imbalances, equalizing opportunities and preventing the marginalization of certain areas considered to be problematic due to the enormous deficits in the basic infrastructure systems supported by the project. The degree of deficiencies in the technical infrastructure in rural areas, the scale of needs and the build up of tasks that need to be undertaken compared with the gminas’ financial capacity makes it possible to conclude that if the Project were not realized, then the elements that form part of the current state of rural infrastructure would lack the measurable effects, which have been obtained as a direct result of conducting the Project. This is confirmed by the opinions of the beneficiaries that indicate that the financial resources obtained from the RDP not only made the realization of particular infrastructure investments easier, but made them at all possible. Selection of the voivodships, in which Component C was implemented Component C was realized in all 16 voivodships. The chosen option of universal (all voivodships and all rural and rural-urban gminas) participation in Component C was a continuation of the analogous strategy within the “ASAL” program (1994-1997), whose principal objective was to provide financial support for the process of building and upgrading rural infrastructure. 5.1.2 Evaluation of the objectives and activities of Component C Evaluation of the objectives of Component C The main objective of Component C consisted in improving and upgrading the infrastructure in rural areas with a view to stimulating private sector investments, facilitating the creation of non-farm jobs and improving the state of the environment and of the population’s health (Operational Manual 2000, PAD, annex 2). On analyzing the program documents and particularly the Project Appraisal Document it may be noted that the scope of the objective, which was allotted to this Component has been broadened. The realization of Component C also aimed to do the following in addition to that described above: 270 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) • to create alternative organizational and institutional structures with a view to ensuring the delivery of sustainable and accessible municipal services, • to introduce a market oriented approach to the process of building and operating infrastructure. All of the recorded objectives were to contribute to the process of finally narrowing the gap between rural and urban areas in the sphere of economic development (PAD page 14). The objectives set for Component C formed part of the achievement of two of the three Project objectives: increasing non-agricultural employment and supporting the process of strengthening local government and regional development. Component activities Component Objectives Project Objectives Figure 52. Diagram of the correlation of the objectives and actions of Component C with respect to the Project objectives Increasing non-farm employment Improvement and upgrading of infrastructure Building and upgrading of roads Supporting the process of strengthening local government and regional development Assisting in building up the institutional resources needed to obtain pre-accession and EU structural funds. Creating alternative organizational and institutional structures (with a view to ensuring the delivery of sustainable and accessible municipal services) Development of the water supply networks Introducing a marketoriented approach to the process of building and operating the infrastructure. Development of the Development of sewage networks and waste management sewage treatment plants system Training and technical assistance Source: own work Adequacy evaluation Taking into account the scale and scope of neglect and the spatial imbalances in the sphere of the development of rural infrastructure and on the other hand the great need to improve the infrastructure provision of rural areas both with respect to the living and working conditions of the rural population and the conditions for creating extra-agricultural areas of activity, the objective set for the Component concerning the building and upgrading infrastructure adequately responds to the above issue. The positive impact of infrastructure investments on improving living and working conditions is obvious and has its own logic. However, it is not applicable for all residents of rural areas to the same degree. The ones in the worst position are those for whom high costs are the barrier to accessing infrastructure services. Despite having the physical possibility, some residents (for economic reasons) partially or fully refrained from consuming newly accessible services (this is confirmed by empirical studies). It is therefore necessary to design infrastructure investments that on the one hand are adequate in relation to the scale of needs and on the other guarantee the delivery of services of appropriate quality and at a price in line with the financial capacity of potential users. “The organizational and institutional structures and the regulatory environment do Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 271 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) not support the efficient delivery of services, which results in unit costs being too high as compared with the finance capacity. The process of building and operating rural infrastructure must have a market orientation to avoid services being subsidized or them being delivered below acceptable standards” (PAD page 8). These important aspects of rural infrastructure development have been reflected in the design of Component C’s objectives. Based on the assumption that “low cost provision of sewer and solid waste services encourages increased subscription by potential users” and on the basis of empirical evidence indicating that consumers protest against high cost services (usually based on over designed facilities – PAD page 3) by withdrawing their subscription, the Project authors introduced additional objectives for Component C. This activity: “training and technical assistance” was supposed foster the creation of alternative organizational and institutional structures with a view to ensuring the delivery of sustainable and accessible municipal services and the introduction of a market oriented approach to the process of building and operating infrastructure. In the circumstances where there are great needs relating to infrastructure development and at the same time where the financial capacity of local authorities is limited (which are “typical” for the Polish countryside), the issue of selecting the investment to be undertaken that is optimal from a financial, economic and social point of view comes to the fore. The “technical assistance and training” activities conducted within the Project were supposed to help with designing the appropriate investments. To conclude, the appropriateness of the objectives of Component C with respect to the problems of the areas, to which it is addressed, may be assessed as satisfactory. Evaluation of the activities of Component C The objectives set for Component C were achieved by way of the following actions: • Developing the water supply network (facilitating the supply of water of adequate quality for domestic and industrial use), • Developing the sewage network and sewage collection and treatment facilities (often realized in conjunction with water supply networks as a way of making up a complete cycle of supplying water of appropriate quality for rural businesses and country residents), • Developing solid waste management systems (including collection, recycling and storage in accordance with the statutory regulations and the sanitary and safety requirements), • Building and upgrading rural roads (consisting in building local feeder roads to motorways under construction and paving surfaces in order to ensure access to certain areas throughout the year irrespective of the weather conditions), • Training and technical assistance (in order to raise awareness of public officials as to the existence of alternative solutions for providing municipal services, improving effectiveness, improving the quality and standard of the provision of municipal services, limiting costs and the required subsidies and reducing costs for the final consumer). Analyses of the level of development of individual fields of infrastructure indicate that the directions of the activities undertaken as part of Component C were appropriate with respect to the type of needs of rural areas. The principle adopted in the RDP of basing on 272 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) expressed demand was the reason why certain types of infrastructure were eliminated from Component C as local authorities had expressed a low level of demand for financial support. The low level of interest of beneficiaries in investments in such sectors as communications or energy (proposals for investment in telecommunications infrastructure in the countryside constituted 0.8% of the entirety of the investment portfolio and the share of power supply projects qualifying for finance represented only 1.3%129) meant that they were not included in the group of activities covered by the support of RDP. The structure of applications approved for implementation broken down according to individual types of infrastructure investments was generally in line with the demand expressed in this respect and met the actual needs relating to the development of technical infrastructure in rural areas. Table 64. Structure broken down according to types of approved and implemented applications Type of investment Percentage of applications Submitted in 2001 Approved in 2001 Roads 54.2 56.3 Sewage systems 28.7 27.7 Water supply systems 16.2 14.9 Solid waste management systems 0.9 1.1 100.0 100.0 TOTAL Source: Own calculations based on data from the Project Coordination Unit Quarterly Reports As indicated by the above information, in the case of one activity the scale of the requirement expressed by the beneficiaries (gminas, associations of gminas, powiats) was not adequate in relation to the level of development of this infrastructure element. The interest in investments concerning the management of solid waste (measured on the basis of the number of applications submitted and also the number of investments undertaken in this sphere) were not correlated with the scale of needs for solid waste disposal sites to be organized in rural areas. This is above all the result of a great need for improvements in the spatial accessibility of rural areas (building and modernizing roads) and water management (water supply, sewage collection and treatment), which in light of the restricted financial capacity of gminas makes it necessary to focus activities and resources on priority investments. It should be noted that for the group of gminas participating in the RDP – the water supply activity - the average indicator of connection to the water supply network that indicated the percentage of homes connected up to the water supply system amounted to 45% and was 12 percentage points lower than for gminas not taking part in the Project. Nevertheless, as regards the sewage network, for gminas implementing investments within the framework of the RDP – the sewage system activity – the percentage of those connected up to the sewage network was slightly higher (by two percentage points), however in the case of this infrastructure element, in light of its low average in rural areas and the minimal scale of the variance, this may be a coincidental distribution. With respect to the density of gmina roads with paved, improved surfaces, gminas participating 129 Ibidiem, p. 14. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 273 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) in the Project – roads activity – were characterized by a lower average density of roads in this category as compared with non-participating gminas (for the former this amounted to 15 km per 100 km2 and for the latter – nearly 19 km/100 km2). Generally, the Project was implemented in areas with a relatively less favorable situation in the spheres under discussion. The evaluation of the adequacy of the last of the activities “training and technical assistance” may be assessed on the basis of the beneficiaries’ opinions concerning their usefulness. None of the training participants assessed the training as not very useful or not useful at all. More than half of the surveyed representatives of the local authorities that confirmed their participation in the training assented that the training had been very useful. In conclusion, the Component actions appropriately respond to the actual problems and needs of rural areas (gminas and powiats). It turned out that it was legitimate to “structure” Component C in such a way as to make it possible for rural communities to choose those infrastructure investments from a list of options whose realization fulfilled their priority development needs as well as help with choosing such investments, which are optimal from a financial, economic and social point of view. Evaluation of participation in Component C Component C was implemented in all 16 voivodships in Poland. In the project documents the main beneficiaries of Component C of the RDP project were considered to be: • Rural gminas, • Urban-rural gminas excluding investments in towns with more than 15000 residents, • Urban gminas excluding investments in towns with more than 15000 residents, • Authorized bodies of inter-gmina associations, • Poviats only in relation to powiat roads, excluding investments in towns with more than 15000 residents. Potential beneficiaries of Component C The potential number of beneficiaries could exceed 2.5 thousand. Rural gminas could potentially constitute the most numerous community of RDP participants, whereas the smallest one would be urban gminas due to the restrictions as to the number of residents (fewer than 15 thousand). The Project was generally targeted at the residents of rural areas. 274 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 65. Number of potential beneficiaries Units Number of potential beneficiaries Percentage Poviats 308 12.1 Urban gminas 78* 3.1 1604 62.6 567 22.2 2557** 100.0 Rural gminas Urban-rural gminas Total Gmina associations No data available *Fewer than 15 thousand residents, ** The number of potential beneficiaries amounted to 2577, and this number of units was invited to training. Differences may result from formal decisions. Source: Rocznik Statystyczny RP, GUS, Warszawa 1999. On the assumption that infrastructure investments designed only within the boundaries of one gmina could lead to the loss of significant benefits due to its small territorial area, the principle of economies of scale was adopted as one of the classification criteria for investment projects in the Project. It was expected that the implementation of the Project would contribute to strengthening the tendency for groups of gminas to undertake common investment activities in the sphere of technical infrastructure. However, the anticipated effects were not achieved by including Gmina Associations (which submitted only 7 applications) in the group of entities authorized to apply for assistance, although projects realized by several gminas were favored. However, the number of road applications received from Poviat Boards indicates that the decision to address the RDP offer to powiats - local government units established relatively recently (we are referring to the start of the Project) would appear to be the right one. On the basis of the number applications received for co-financing submitted by entities authorized to do so (and also the level of the amounts in relation to which co-financing has been applied for) it is possible to unequivocally state that the interest in participating in RDP was very large. At total of 2077 applications were submitted (Quarterly Report No. 3). This means that theoretically (assuming that each entity had submitted only one application) 81% of potential beneficiaries submitted a requirement for funding from RDP for undertaking infrastructure investments. The finance required for implementing investment tasks applied for by gminas was five times greater than the anticipated amount covered by the Project’s financial capacity. If one looks at the distribution of the number of applications (as well as their value) that were submitted by individual voivodships it is possible to note that generally this is case of a simple relationship – the lower the level of the infrastructure provision, the greater the scale of the investment finance requirement. In light of the very large variation between voivodships with respect to the number of administrative units they comprise, the amount applied for and the number of applications was referred to the number of gminas in a given voivodship. In light of such a synthetic measure the following voivodships expressed the greatest needs: Świętokrzyskie, Podkarpackie, Lubelskie, Małopolskie, Łódzkie (all of which were characterized by a low standard of technical infrastructure). In these voivodships the average amount applied for and the number of applications per gmina was the highest. The lowest interest in Component C (measured on the basis of Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 275 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) the number of applications for finance submitted) was noted in the Opolskie, Śląskie, Zachodniopomorskie and Dolnośląskie voivodships. Table 66. Applications submitted for financing in Component C Voivodship Value of Number of applications for Number of applications applications for co-financing in the ranking submitted (PLN co-financing list thousand) Level of technical infrastructure (synthetic indicator)* Podkarpackie 96 425 153 32 Low Dolnośląskie 64 221 54 5 Medium Świętokrzyskie 63 052 137 64 Very low Lubelskie 92 915 168 29 Low Małopolskie 77 231 185 41 Low Warmińsko-Mazurskie 44 178 85 37 Low Pomorskie 48 087 78 15 High Zachodniopomorskie 23 323 53 25 High Śląskie 31 557 39 7 High Wielkopolskie 40 283 152 49 High Opolskie 20 989 38 4 High Lubuskie 28 198 52 8 Low Łódzkie 77 003 184 34 Low Mazowieckie 97 879 135 56 Very low Podlaskie 35 396 77 35 Very low Kujawsko-Pomorskie 31 080 93 35 Medium TOTAL 871 816 1682 476 - * synthetic indicator - own calculations based on BDR data 1999, GUS, Warszawa Source: Quarterly Report of the Project Coordination Unit no. 3 Beneficiaries of Component C The largest number of beneficiaries was located in the Świętokrzyskie voivodship, then in the Wielkopolskie, Małopolskie, Mazowieckie and Kujawsko-Pomorskie Voivodships. The smallest number of beneficiaries was situated in the Śląskie, Opolskie, Dolnośląskie and Lubuskie voivodships (apart from the Lubuskie voivodship, these voivodships are in the relatively most favorable position as regards technical infrastructure). Rural gminas constituted the largest proportion of beneficiaries – with them making up 61% of the total, whereas urban gminas represented the smallest share, as these only represented 8% of the total. Thirty percent of road investments were undertaken on the initiative of powiats. If the activity “training and technical assistance” is taken into account, the Project encompassed more or less all the potential beneficiaries, which is in line with its guiding principles. At least one of the possible infrastructure investments was realized within the 276 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) boundaries of 25% of beneficiaries and it was a frequent occurrence for gminas to carry out two different types of infrastructure investment each. Figure 53. Spatial distribution of gminas participating in the RDP (all activities) Source: MIS Database The largest number of water supply investments was undertaken in the Świętokrzyskie, Mazowieckie and Małopolskie voivodships. The greatest number of investments relating to sewage was realized within the Podkarpackie, Kujawsko-Pomorskie and Wielkopolskie voivodships. The Świętokrzyskie voivodship was the definite leader when it came to the number of road contracts. A few investments with respect to solid waste management systems were undertaken within only 3 voivodships: the Mazowieckie (two investment projects), Wielkopolskie and Lubelskie (one each) voivodships. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 277 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 67. Number of contracts in Component C broken down according to voivodships Type of investment project Roads Sewage System Water Supply System Solid Waste TOTAL Percentage of Contracts Dolnośląskie 7 5 3 0 15 2.1 Kujawsko-pomorskie 32 22 4 0 59 8.2 Lubelskie 32 13 1 1 47 6.5 Lubuskie 8 11 3 0 22 3.0 Łódzkie 42 7 1 0 51 7.1 Małopolskie 34 20 14 0 68 9.4 Mazowieckie 36 4 17 2 59 8.2 Opolskie 9 4 2 0 15 2.1 Podkarpackie 23 27 2 0 52 7.2 Podlaskie 40 3 6 0 49 6.8 Pomorskie 20 6 2 0 28 3.9 Śląskie 7 3 0 0 12 1.7 Świętokrzyskie 62 12 17 0 91 12.6 Warmińsko-mazurskie 24 14 7 0 46 6.4 Wielkopolskie 31 22 7 1 61 8.4 Zachodniopomorskie 19 20 9 0 48 6.6 TOTAL 428 194 97 4 723 100.0 Voivodship Source: Quarterly Report of the Project Coordination Unit no. 17 The analyses that have been conducted lead one to reflect upon the purposefulness of dispersing the resources over all 16 voivodships – would it not rather be more effective to concentrate them in the voivodships where the scale of infrastructure neglect was the greatest and therefore those that had expressed the largest requirements with regard to financial support? Would this type of strategy not make a greater contribution to leveling out the differences between individual regions? And is distributing the funding over all the voivodships not an action that could lead to a widening of spatial imbalances with respect to infrastructure development and thus indirectly the conditions for conducting business and living conditions of the rural population? It is very difficult to make an unequivocal assessment of this matter. An argument in favor of making available financial resources to all of the voivodships is the fact that all voivodships (irrespective of the average level of technical infrastructure) are characterized by a larger or smaller degree of internal variance – this is a specific characteristic of rural areas in Poland. Secondly, in all of them one notes the existence of the problem connected with the underdevelopment of the sewage network (including the sewage treatment system), which has a particularly unfavorable impact on the natural environment and on the living conditions of the rural population in all country regions in Poland. Another problem consists in the fact that in all voivodships the majority of rural gmina roads are generally of very poor quality. 278 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Not limiting oneself to specific regions (voivodships) with respect to selecting beneficiaries was also the continuation of the relevant strategy in the rural infrastructure finance project forming part of the “ASAL” program (1994-1997). The chosen option of universal participation in the RDP in Component C gave all units equal opportunities to access resources and at the same time ensured that the most efficient infrastructure investment projects in the country were selected and carried (in the context of objectives associated with creating new jobs, stimulating economic growth and improving the state of the environment). And this may well be the most important argument confirming that it was right to select beneficiaries in this way, especially in a situation where the amount of financial resources was relatively low as compared with needs. To summarize, it should be stated that the spatial range of the Project was considerable. Carrying out Component C in all 16 of Poland’s voivodships should be considered as being justified. Evaluation of the relevance of Component C to government policy The purpose of Component C was in line with the objectives of the State concerning rural areas. In all the government documents mentioned below, the development of rural infrastructure as well as the issues associated with creating new jobs in the countryside is considered to be the highest priority objective of the rural development strategy: • Strategy for agriculture and rural areas (1990); • Assumptions of socio-economic policy for rural areas, agriculture and the food economy (1994); • Medium-term strategy for the development of agriculture and rural areas (1998) • Pact for agriculture and rural areas (1999) • Coherent structural policy for the development of rural areas and agriculture (1999) The purpose of Component C is fully in line with the general principles of state policy towards rural areas. 5.1.3 Situation before and after the implementation of Component C Efficiency evaluation of Component C According to the Project Appraisal Document (PAD) the actions undertaken within the framework of Component C were supposed to bring about the following (PAD Annex 2): • Realization of investment projects of a value of approximately 177.39 million USD with respect to modernizing and building infrastructure; • Improving the quality and efficiency of municipal services in the countryside. The type of indicators adopted in Component C, which were used to measure the outcomes of the infrastructure investments undertaken in rural areas, was proposed in the basic documents drawn up in March 2000: • Project Appraisal Document; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 279 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) • Operational Manual. The table below shows the output indicators: Table 68. Output indicators according to the Project Appraisal Document and the Operational Manual Project Appraisal Document Water and sewage systems: Operational Manual Water supply and sewage disposal system: Amount of treated water produced Amount of water supplied / sewage collected Number of water connections Number of water connections Number of sewer connections Number of connections to the sewage system Receivable collection rate Bill collection rate Tariff levels Tariff levels Losses (% of production) Level of losses (as a % of water supplied / sewage collected) Operating expense Operating expense Roads Roads sector Length of roads upgraded or built km of roads upgraded or built Average daily traffic intensity Average daily traffic Operation and maintenance costs Operation and maintenance costs Number of road accidents Number of accidents Solid waste sector: Solid waste sector: Solid waste disposal capacity Solid waste collection capacity Landfill capacity Landfill capacity Operation and maintenance costs Operation and maintenance costs Number of operators trained Number of projects, which received technical assistance Source: PAD and Operational Manual The following indicators were shown in the monitoring table: Concerning water supply systems: • Number of water treatment stations; • Number of water supply connections - households; • Number of water supply connections – public buildings, business entities and others; • The length of the installed water supply system. Concerning sewers and sewage treatment plants: 280 • Number of sewage connections – households; • Number of sewage connections – public buildings, business entities and others; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) • The length of the installed sewage system; • The number of sewage treatment plants built; • The efficiency of sewage treatment plants built. Concerning roads: • The length of roads built or upgraded in km. Concerning solid waste management systems: • Number of landfills built; • Capacity of landfills; • Number of users - households; • Number of users - others. Concerning training and technical assistance: • Number of trained technical staff of JST, WBWP and RO ARIMR employees; • Number of organized training courses, • Number of projects that have received TA support Concerning all activities • Jobs created (both directly and indirectly). It may be generally stated that the indicators established for evaluating the program outputs diverge from the planned values, although the scope of variances differs for individual Program outputs. The considerable diversity of results amongst voivodships is also worth noting. In the case of Component C one can observe the whole range of possible interrelationships between assumed and achieved results: assumed outputs were lower than actual; assumed values exceeded planned ones and the third type of relationship the results achieved were in line with the planned ones. Average indicators for actions connected with developing the water supply and sewage networks and building and upgrading roads did not reach the planned values. It should be noted that the relatively largest imbalance between planned and achieved values concerned the sewage system. Only indicators for services connected with training considerably exceeded the planned values. We have presented below a characterization of the outputs for individual activities. Water supply system The anticipated result of the actions connected with developing the water supply system was building 886.6 km of water supply network, to which 13,086 households and 1,467 other facilities (public utility facilities, business entities and others) were supposed to be connected in accordance with assumptions. With respect to the length of the water supply network higher (by 4.1%) results than those anticipated were achieved. Nevertheless, it was not possible to attain results of such a scale in other areas, concerning e.g. the number of connections. This is a more important element in that a considerably lower Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 281 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) number of water supply connections means that there is a lower number of users of a particular service. The total number of connections for households and business entities was 1235 lower than the planned figure, with the assumed indicator value for households being only 1.7% lower than the actual one. Taking into account the small scale of the divergences between the assumed and actual numbers of built connections to the water supply system it is possible to consider that the objective had been fully achieved to this extent. The scale of imbalances was a lot greater for other facilities. Nearly 70% fewer facilities of various types were connected up to the water supply system than had been assumed. The total number of connections for households and business entities was 8.5% lower than assumed. The number of water treatment plants was exactly the number that had been planned. Almost 40% of the new jobs planned were created. Table 69. Planned and achieved results in the area of activity: the water supply network Indicator Type Measure Achieved Planned Length [km] 923.0 886.6 Number of water supply connections households Number [units] 12 869 13 086 Number of water supply connections – others Number [units] 449 1 467 Number of water treatment stations Number [units] 14 14 Number of jobs created Number [units] 821 2 065 Length of water supply system installed Source: Based on MIS –C data Sewage system With respect to this area of activity it was assumed that 916 km of sewage network was to be built, which 24.8 thousand households and 1.88 thousand other facilities (public utility facilities, business entities and others) were supposed to use. The scope of the divergences between the assumed and actual number of kilometers of sewage network built were considerable: the length of the built sewage system was 13.2% greater than the planned figure. Sixteen percent fewer sewer connections to households and 24.5% to other facilities were built than the assumed level. In the case of the former the number of connections was 3950 lower and for the latter 462 lower than the planned amount. The total number of connections, which were established as part of the RDP was therefore 4412 (16.8%) lower than the planned figure. A total of 236 sewage treatment plants was built, which was 115 fewer than planned. In the case of this infrastructure element 61% of the plan had been achieved. Assumptions concerning the total efficiency of sewage treatment plants built proved to be higher than the attained result. The accomplished task was more or less in line with assumptions despite the fact that the number of sewage treatment plants put into operation was lower. As regards jobs, 1238 were created (i.e. 23% of the assumed number). 282 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 70. Planned and achieved results in the area of activity: the sewage network Type Indicator Measure Achieved Planned Length [km] 1037 916 Number of sewer connections - households Number [units] 20 850 24 800 Number of sewer connections – others Number [units] 1 422 1 884 Number of sewage treatment plants built Number [units] 236* 351 Length of sewage system installed 3 Efficiency of sewage treatment plants built Efficiency [m / per day] 14 853 12 122 Number of jobs created Number [units] 1 238 5 400 * including 213 farm sewage treatment facilities Source: Based on MIS –C data Building and upgrading roads The total length of roads built and upgraded amounted to 734 km and was 177 km longer than the assumed figure. The attained result was therefore 32% higher than the planned one. Within this area of activity 3704 new jobs were successfully created (i.e. 24.5% of the assumed number). Table 71. Planned and achieved results in the area of activity: building and upgrading roads Indicator Length of roads built and upgraded Number of jobs created Type Measure Achieved Planned Length [km] 734 557 Number [units] 3 704 15 100 Source: Based on MIS –C data Solid waste management system Solid waste management is the area of activity that evoked the least interest on the part of beneficiaries and the assumed outputs were the lowest for this. At the same time, this is the only area of activity, in which the divergences between assumed and actual outcomes with respect to the number of established facilities were slight. Four landfills were set up with a total capacity of 138.9 thousand dm3. Their capacity was nearly 35% lower than the planned figure. Of the areas of activity described hitherto, solid waste management was the first one, in which the number of actual users exceeded that of planned ones. It was planned that a total of 16.9 thousand users (including 96.8% of households) would use the four newly established landfills, whereas in reality the figure was 6.1 thousand (36%) higher than the assumed level. Of the actual users of solid waste disposal sites, the increase in users designated as “others” was decidedly higher than the planned figure (the share of households in actual users amounted to 80.7%). It was assumed that 240 jobs would be created within this area of activity of which 74% are already functioning. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 283 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 72. Planned and achieved results in the area of activity: solid waste management system Type Indicator Measure Achieved Planned Number of landfills built Number [units] 4 5 Number of users - households Number [units] 18 610 16 395 Number of users - others Number [units] 4 464 542 Landfill capacity (m3) Capacity [m3] 138 926 200 000 Number of jobs created Number [units] 177 240 Source: Based on data from the MIS database Training of technical staff and technical assistance for gminas, ARMA and VPIU. In 2001 implementation of the training program was commenced. Ninety six training courses were supposed to be conducted, which at least 3,000 representatives of gmina, powiat and voivodship authorities were to attend. Ninety seven training courses were conducted, in which 3019 persons participated, therefore 19 more than planned. According to the assumptions, all potential beneficiaries numbering 2,577 were invited to take part in training. In this area of activity, it should be considered that the assumed results were in line with those actually achieved. Table 73. Planned and achieved results in the area of activity: training Indicator Definition Number of Regional SelfGovernment Unit technical staff, VPIU and ARMA Regional Office employees trained Result: Achieved Planned Number of Regional SelfGovernment Unit technical staff, VPIU and ARMA Regional Office employees trained by the RTI 3 019 3 000 Number of courses organized In accordance with contract with RTI 97 96 Number of Regional SelfGovernment Units invited to training (including participants) All Regional Self-Government Units entitled to participate in the RDP will be invited 2 577 2 577 Source: Project Coordination Unit Report We have noted a similar situation for technical assistance. In accordance with the assumptions, all gminas that had applied for co-financing received technical assistance. One hundred percent of set plans were achieved in this area of activity. 284 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 74. Planned and achieved results in the area of activity: technical assistance Indicator Percentage of projects receiving TA Definition All gminas that had applied for subsidies (2077), ARMA Regional Office and VPIU should receive TA Result: Achieved Planned 100% 100% Source: Project Coordination Unit Report It should be noted that an on-line course was developed under Component C (by RTI and Lemtech Konsulting specialists) based on the guide entitles Execution of Infrastructure Projects in Rural Gminas. It provided all the interested persons with an opportunity of self-education and a possibility of verifying the knowledge acquired at school. The course could serve as an independent tool aiding the preparation of infrastructure investment projects. The access time to the training on the PCU Web site was unlimited. There were no limitations as to the multiple use of the course. In the course the information contained in the guide was divided into four training modules summing up its individual chapters. 1. Stages of Project execution and identification; 2. Project technical and technological analysis and calculation of the current and future value of cash (constituting an introduction to the Project financial and economic analysis); 3. Project financial installation and implementation; 4. Project financial and economic analysis. The number of persons visiting the site (exceeding 5000 confirmed visits) may show a great interest in that completely new tool. The obvious advantage of such solution was the availability of a guide containing a complete base of knowledge concerning that subject. Diverse impacts were obtained compared with intended values in individual areas of activity. The greatest variances concerned the number of new jobs, which were supposed to be created as a result of realizing individual investment projects. However, there is a proviso that in this case the final deadline for “evaluating” the impact in this respect is in the years 2006- 2009 (five years after investment completion). Training and technical assistance as well as building and upgrading of roads were those areas of activities where there was the greatest concurrence of planned and achieved results. In contrast the area of activity – solid waste management was characterized by the highest efficiency - with the number of end users of the provided service being higher than the assumed level. The lowest efficiency was observed in the area of activity – the sewage network. The total number of connections to the sewage system was 32% lower than the assumed figure and 16% lower in the case of the water supply system. Despite the various outcomes in individual areas of activity, the Project’s efficiency can be assessed as satisfactory. Higher than expected results were attained in the case of Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 285 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) the growth of the length of the water supply network, the sewage system, the length of built and upgraded roads and the efficiency of sewage treatment plants. Criteria for financing in Component C The level of co-financing using RDP funds was contingent on the type of investment planned to be implemented and could amount to a maximum of 50%. Table 75. The level of subsidies for individual types of infrastructure investments Maximum subsidy share (% of the costs of the project) Actual average subsidy share Water supply system 30.0 26.0 Sewage system 50.0 35.0 Solid waste 50.0 35.0 Gmina roads 50.0 44.0 Poviat roads 50.0 44.0 Type of investment project: Source: Project Coordination Unit Report Regional Steering Committees ranked the submitted projects according to the following criteria: • The importance for the broader regional economic development plan; • The potential for direct and indirect job creation; • Joint proposals submitted by several gminas, which reflect the principle of economy of scale in Project design; • The contribution of local government to financing investment costs; • The economic rate of return.130 ARMA RD’s verified the quality of the investment proposals on the basis of technical, economic, financial and institutional requirements. According to the Project assumptions, infrastructure projects were realized in areas with high population concentration. To summarize, it is possible to state that the criteria for co-financing were set out unequivocally. The fact that during the assessment of the investment projects additional points were awarded for matching funds that were greater than those that had been assumed in the criteria encouraged gminas to design investments with a greater share of own resources than that which was required. In practice, this was translated into an average threshold for co-financing that was lower than the planned one for each area of activity. This consequently made it possible to allocate resources to additional investments and finance additional projects. However, an excessive emphasis (a larger number of points in practice) on new jobs being created as a result of the investments could “tempt” the gminas to declare a larger 130 RSCs could adjust the project evaluation criteria recommended by BS to regional priorities and to assign various ranks to these criteria. 286 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) number of jobs than the number that was actually possible to attain. This was to a certain extent an area where points could be „fought over”, which in light of the strong competition of good quality projects was particularly important and in the end could have been a decisive factor for the points total and therefore for the final outcome. It should be noted that no other program currently in operation (SAPARD, PHARE 2000) has such a criterion. We consider it appropriate for the share of co-financing from the RDP to be set lower for the water supply area of activity than for the others. This criterion is justified by the fact that the development of this infrastructure element was most advanced in rural areas (in relation to other areas covered by RDP financial support). The acceleration of the process of building the water supply system without parallel activities pertaining to the sewage network could entail a widening of the imbalances that are already very large. However, in circumstances where imbalances are increasing, criteria that rewarded activities relating to the water supply system, which were conducted in parallel to the development of the sewage network would be optimal. Evaluation of the utility of Component C The utility of the project was analyzed from the point of view of the overall contribution of the Project to the development of rural infrastructure and thus on a nationwide scale and from the point of view of the outcomes attained as a result of its implementation in the units that had participated in the RDP in relation to the results in the group of gminas that had not participated in the Project. Utility, which was also analyzed from the point of view of evaluating the impact of the Project on alleviating (or maybe aggravating) the problems of infrastructure provision in rural areas, above all concerned the reduction of the imbalances between the water supply network and the sewage system. Water supply network Within the framework of the first of the areas of activity in Component C a total of 923 km of water supply network was built, which is used by 13,291 users (we treat them as being equivalent to the number of connections). The annual growth of the water supply network realized under the RDP over the duration of the Project constituted 5.2% of the annual increase in the entire length in rural areas. The average annual growth rate of new consumers compared with the whole amounted to around 4.7%. Generally, without the Project’s contribution the length of the water supply network and the number of connections would have been lower by the measurable outcomes that were brought about by the Project. The scale of the achieved outcome was small with respect to the growth of the water supply network and the number of newly created connections in general in rural areas. A proper evaluation of the contribution of the Project in this regard should however take into account the fact that only 4.0% of the total gminas in the country (of which more than 60% were rural gminas) participated in this area of activity. Bearing in mind this fact, the measurable outcome attained in this sphere should be assessed as satisfactory. The growth of the length of the water supply system as well as the number of new users established within the framework of the Project was more or less in proportion (it was even slightly higher in the case of the number of connections) to the overall outcomes achieved nationwide in the years when the RDP was in progress. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 287 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 76. Change in the value of indicators for water supply infrastructure YEAR Indicator describing Change in the years 2000-2003 Average annual growth rate 2000 2003 The length of the water supply network (in thousand km) 161.8 179.5 17.7 5.9 Including those set up within the framework of RDP (in thousand km) 0.0 0.923 0.923 0.308 Number of water supply connections (in thousand units) 2 297.3 2 577.8 280.5 93.5 0.0 13.318 13.318 4.430 Including those set up within the framework of RDP (in thousand units) Source: Data from BDR 2003, GUS, Warszawa; Data from the MIS database. In the group of gminas participating in the RDP, over the duration of the Project the number of water supply connections increased by 13,318. The outcomes attained as a result of realizing the RDP constituted 90% of the general increase in water supply connections in the group of gminas that were participating in this activity. In gminas that were participating in the Project the results obtained should be assessed as very positive. The outcome practically achieved in this sphere could be “credited” to the RDP. Table 77. Change in the value of infrastructure indicators in the group of gminas participating and not participating in the RDP Gminas Indicators Year Participating in the RDP Not participating in the RDP Average number of connections to dwellings and communal residential buildings per one gmina 2000 880 1103 2003 1078 1228 Difference in 2003-2000 198 125 % change (2000 =100%) 22.5 11.3 Source: Data from BDR 2000, 2003, GUS, Warszawa An additional point contributing to the positive assessment of the utility of this area of activity is justified by the fact that the rate of change relating to the average number of water supply connections in the group of gminas participating in the RDP was two times lower than in the group not participating in it. Sewage network and sewage treatment plants Within the framework of the second activity in Component C a total of 1037 km of sewage network were built, which are used by 22,3 thousand consumers. The above outcomes achieved as a result of conducting the Project constitute the Project’s real contribution to the development of the sewage system in rural areas. The annual growth of the sewage network system attained as part of the RDP over the duration of the Project represented 8.1% of the annual increase in the entire length in rural areas. The average annual growth rate of new consumers compared with the whole 288 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) amounted to around 11.7%; therefore this is nearly two and a half times greater than in the case of the first area of activity. The scale of the achieved outcome is already considerable with respect to the growth of the sewage network and the number of newly created connections in general in rural areas. A proper evaluation of the contribution of the Project in this regard should however take into account the number of the units participating in this area of activity as in the case of the above area of activity. Investments in sewage systems (including sewage treatment plants) were carried out in the territory of 7.5% of the total number of gminas in the country. Bearing in mind this figure, the measurable outcomes attained with regard to sewage networks could be assessed as satisfactory. The growth of the length of the water supply system as well as the number of new users established within the framework of the Project was higher than the outcomes achieved nationwide in the years when the RDP was in progress (taking into account the ratio of participation to the achieved outcome). The throughput capacity of the sewage treatment plants put into operation within the framework of the RDP (numbering 236, of which 213 were farm sewage treatment facilities) represented 6.6% of the total increase in the total throughput capacity of sewage treatment plants over the duration of the Project. Table 78. Change in the value of indicators for the sewage system and sewage treatment plants YEAR 2000 2003 Change in the years 2000-2003 The length of the sewage network (in thousand km) 16.2 28.8 12.6 4.2 Including those set up within the framework of RDP 0.0 1.037 1.037 0.34 Number of sewage connections (in thousand units) 259.4 450.1 190.7 63.6 Including those set up within the framework of RDP 0.0 22. 272 22. 272 7.424 Throughput capacity of sewage treatment plants (in m3/per day) 787 518.2 1 010 892 223 373.8 74 457 Including those set up within the framework of RDP 0.0 14 853 14 853 4 951 Indicator Average annual growth rate Source: Data from BDR 2003, GUS, Warszawa; Data from the MIS-C database. In the group of gminas participating in the RDP, over the duration of the Project the number of sewage connections increased by 33.4 thousand, of which 22.3 thousand had been built within the framework of the Project. The outcomes attained as a result of realizing the RDP constituted 67% of the general increase. In gminas that were participating in the Project the results obtained were higher than those of gminas not participating in the RDP as shown by the data below, which means that the utility of the RDP should be given a highly positive assessment. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 289 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 79. Change in the value of infrastructure indicators in the group of gminas participating and not participating in the RDP Gminas Indicators Year Participating in the RDP Participating in the RDP 194.3 114.6 385.1 184.1 Average number of connections to 2000 dwellings and communal residential 2003 buildings per one gmina Difference in 2003-2000 % change (2000 =100%) 69.5 60.6% Source: Data from BDR 2000, 2003, GUS, Warszawa From the point of the view of the utility of the Project it is important to evaluate its impact on solving basic problems relating to rural infrastructure and in this specific case we are considering the issues associated with the water and sewage system. An important element of the assessment consists in analyzing the impact of the activities undertaken within the framework of the Project on changing the imbalances between the number of water supply and sewage connections. This would make it possible to fully assess the Project’s real contribution. The data summarized in the table below show the tendencies with regard to the magnitude of the impacts including and excluding the contribution of the RDP and also the significance of the Project’s contribution to eliminating the differences in the sphere under discussion. If the result attained with the support of RDP co-financing is subtracted, the difference between the number of water supply and sewage connections would be 9,403 lower than the figure that it was finally possible to achieve. In practice this means that the Project’s activities have contributed to a reduction in the imbalances between the number of water supply and sewage connections nationwide and consequently the Project has had an impact on improving the state of affairs relating to the water supply and sewage system, thus indirectly on the improving the living conditions of the rural population. If it were not for the Project’s contribution, the imbalance in this respect would be 10.5% greater than the actual one. 290 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 80. Changes in the imbalances between the number of water supply and sewage connections Year Impact excluding the Project’s activities Change in the years 2000-2003 (in %) 2000 2003 Change in the years 2000-2003 Number of water supply connections (in thousand units) 2 297.3 2 577.8 280.5 268.3 4.3 Number of sewage connections (in thousand units) 259.4 450.1 190.7 169.4 11.2 2 037.9 2 127.7 89.8 99.2 10.5 Number of water supply connections established within the framework of RDP (in thousand units) 0.0 12. 869 12. 869 - - Number of sewage connections established within the framework of RDP (in thousand units) 0.0 22.272 22. 272 - - Difference 0.0 -9.403 -9.403 - - Indicators Difference Source: Data from BDR 2003, GUS, Warszawa; Data from the MIS database. Despite the low proportion of gminas in the areas where this activity was carried out, the Project’s contribution to the development of the sewage system and reducing the imbalance between the number of water supply and sewage connections should be assessed as very highly satisfactory. Building and upgrading roads With respect to the road network, unlike in the case of the water supply and sewage network it is not possible to analyze the Project’s contribution to the overall achievements in this area in the same way for formal reasons. The Project indicator adopted for evaluation purposes for both roads built and upgraded treats these two categories jointly and does not divide roads in accordance with GUS standards. On the other hand, generally available statistical data for rural areas divide roads into various categories, which have not been taken into account in the Project monitoring. However, the most important barrier to conducting this type of analyses proved to be the basic change in the category of roads and the aggregation by GUS of data concerning roads after 2000 according to other categories than those that had been applicable prior to this period. For this reason it is not possible to assess the changes relating to the growth of gmina roads and the increase in upgraded roads. This situation indicates that with respect to designing indicators for evaluating the Project it was legitimate to take into account the accessibility and scope of indicators, which make it possible to compare the results attained in the groups of units participating and those not participating in the Project and evaluations. Solid waste management With regard to solid waste management, we are dealing with an analogous situation to the above, i.e. for formal reasons it is not possible to analyze the Project’s contribution to the overall achievements in this sphere. The Project indicator adopted for evaluation purposes concerns the number of landfills and their capacity in cubic meters, whilst the Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 291 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) information available for rural areas concerns the area of landfills in ha. Thus this case also confirms the need to design or select such Project monitoring indicators that make it possible to evaluate its contribution and which were therefore commensurate with generally available indicators. The utility of the Project should therefore be assessed as highly satisfactory. 5.2 Bottom – up analysis 5.2.1 Identification and measurement of Component C costs and other expenditure The budget allocated to the Component C was the biggest item within the whole RDP budget. It accounted for about 65 percent of the whole budget. Its share was over twice as big as for all Sub-Components B (B1, B2, B3) taken together. Table 81. Component C share in the RDP budget Item Component C Planned cost in USD million 194.89 Percentage share in whole RDP 64.% Bank financing (in USD million) 57.17 Percentage share of Bank financing 47.6% Source: PAD, p.15 In the Project Appraisal Document the envisaged total budget for Component C, covering all its elements (including training programs and other), amounted to USD 194.89 million. The planned share of costs of investment tasks in the overall Components budget accounted for 91.0 percent for Component C. The Component C budget is presented in the table below. Table 82. Component C budget Bank financing (USD million) Self-governments (USD million) Total costs (USD million) Rural infrastructure investment projects 54.54 122.85 177.39 Water and waste water treatment plants 27.00 60.10 87.10 Roads and solid waste 27.54 62.75 90.29 Training program for technical staff 1.11 - 1.11 Technical assistance for VPIU staff and local government officials 1.09 - 1.09 Evaluation study 0.12 - 0.12 - 14.87 14.87 57.17 137.72 194.89 Item Variable costs (O&M) Total costs Source: PAD, p. 77 292 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Based on the quarterly report no.17 concerning the Project management, the total utilization of loan funds and co-financing by the Polish side from the start of the Project in comparison with the plan covering the whole duration of the Project amounts to 87.96 percent, and the value for Component C is 84.54 percent.131 The table below shows that the actual share of local governments in financing infrastructure investment projects was smaller than planned. Table 83. Forecasted and actual investment tasks financing breakdown Item Bank's funds Local government Total Forecasted share 31.0% 69.0% 100.0% Actual share 38.0% 62.0% 100.0% Source: Operating manual, PAD, PCU 2005 data The actual costs of execution for Component C have been lower than the planned costs. The difference between the planned costs and actual costs is, among other things, due to fluctuations in the USD exchange rate. While planning the costs, the exchange rate of PLN 4.27 per USD was assumed. However, during the execution of the tasks the USD exchange rate decreased significantly (in 2003 the average rate amounted to PLN 3.89, and in 2004 the average rate was PLN 3.65). From the financial point of view sewerage and roads are the biggest tasks, which together account for 88.6% of total funds spent on investment tasks. The expenditure for waste disposal sites had the smallest share in the budget, the cost of execution of these sites was equal to 1.2% of total investment expenditure. Additional costs Each gmina incurred costs connected with preparation of investment projects; this activity included preparation of technical documentation, designs, costs connected with clarification of status with regard to land ownership issues. However, those are the costs which are normally incurred by gminas when executing an infrastructure project. In local government officials' opinion costs connected with the required transport and telephone calls are not a significant burden on gminas' budgets. 5.2.2 Identification and measurement of the Project's products and effects Within Component C a number of investment projects have been executed; these investment projects contributed to an improvement of situation with regard to infrastructure facilities in the areas covered by the Project with regard to road network, sewerage, sewage collection and treatment facilities and solid waste management. Water supply system In total 923 km of water supply systems have been built (or modernized) and 12 water treatmemnt plants have been built. 131 According to latest data obtained directly from PCU (17.05.2005), the degree of the World Bank’s loan absorption inreased to 100.0% of the planned level in case of Component C. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 293 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Sewage collection and treatment system Under the RDP 213 new farm sewage treatment plants in a single gmina having approx. 5.5 thousand inhabitants have been built and 23 municipal sewage treatment plants have been built or modernized. Waste disposal sites 4 waste disposal sites have been built with a total capacity 138 926 m3. Roads Within Component C roads with a total length of 734 km administered by powiats and gminas have been built and upgraded. Training for technical staff As part of this activity 97 courses of training have been conducted in which 3019 persons participated. In accordance with the Project's objectives all potential beneficiaries (2577 persons) have been invited to take part in the training. External effects The conducted surveys have shown that the implementation of infrastructure investment projects within Component C has had a wide-ranging influence at the level of gminas, individual inhabitants, as well as business persons in rural areas. The main external effects of Component C include: • • • 294 Improved living conditions of rural population; this concerns: o changes in standard of durable goods used by households, o improved technical standard of housing in rural areas, o facilities making work easier in a household or a farm, o improved health of rural population, o better transport conditions in local systems, o improved conditions of transport for children to/from school, o improved safety for rural population, o “potential” reduction of operating costs (especially in case of sewerage). Improved conditions for conducting business operations; this concerns: o better standard of equipment used by businesses, o better chances for businesses to survive in the market, o better spatial access. Growth in business activity; this concerns: Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) • o new accommodation in agritourism farms, o new business entities (small, mainly commerce-related), o seasonal jobs connected with repair and modernization of buildings, o greater demand for building materials and materials applied in repairs. Increased investment attractiveness of a gmina; this concerns: o greater interest in purchase of building lots, o greater interest in purchase of lots for recreation purposes. • Stronger social activity and social integration at local level. • Improved competence and greater experience of local government staff in planning (in accordance with market economy orientation) and execution of infrastructure investment projects. The surveys have revealed a number of examples of positive influence of the infrastructure investment projects; some are quite specific. The scope of the influence differed, depending on the activity. The investment projects connected with development of water supply and sewerage systems had the greatest positive influence on improvement of living conditions and improvement of conditions for conducting business operations. The opinions expressed by local government officials imply that greater awareness of the needs of the rural population in the sphere of infrastructure improvement in their localities and a strong impulse to work jointly for the benefit of local communities can be mentioned as the positive influence resulting from such investment projects. In many interviews formation of certain patterns and modes of behaviour as a result of improved living conditions was also indicated. In the light of survey results (questionnaires and interviews) and overall analysis of the collected opinions the external effects accomplished as a result of execution of investment projects as part of Component C can be judged very positively. Practically, we have not encountered a completely negative assessment of the investment projects carried out. All respondents said that the investment projects were useful or very useful, and that the funds obtained from the RDP were instrumental in executing a large part of those investment projects. Very good cooperation with the ARMA Regional Office and VPIUs in the opinion of local government officials, mainly with regard to competence of the staff and their usefulness in practical aspects at the phase of preparing an application and project execution has brought about acquired experience resulting in greater activity of gminas in execution of investment projects using other funds from aid funds sources. The Project and its high quality structure (this means the cooperating units) has contributed to strengthening of the process of formation of new “bodies”, i.e. teams of persons who specialize in project implementation in local governments. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 295 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) 5.2.3 Subjective evaluation of Component C according to the beneficiaries (based on the opinion of local governments, business persons, and inhabitants of rural areas) Evaluation of objectives and activities of Component C with regard to the beneficiaries' needs Evaluation of the role of financial support in the development of rural infrastructure Among main problems affecting the development of infrastructure in rural areas the lack of funds is mentioned most strongly by local authorities. Nearly 77% of the surveyed local government officials pointed out that factor (geographical conditions of gminas were indicated in the second place by 16% of respondents, followed by lack of structure plans indicated by 5% of respondents). The significant difference between the number of respondents who indicate lack of funds and the next mentioned barrier for development of infrastructure confirms the importance of the first obstacle, and at the same time points out the importance of the problem, i.e. limited financial possibilities of local governments. The results of surveys covering local government officials in gminas participating and not participating in the RDP are one of the bases for evaluation of the financial support and role of the RDP in the process of rural infrastructure creation. Nearly a half (48%) of investment projects in the surveyed gminas would not be executed at all without the funds available from the RDP. In case of 74% of the gminas lack of financial support would cause a delay in the investment execution. If the funds from the RDP were not available, it would be necessary to divide 58% of the surveyed investment projects into stages. Taking the average value for answers to all questions asked below, in the opinion of local government officials without the financial support provided by the RDP only 42 percent of the infrastructure investment projects would have a chance to be executed in accordance with the plans with regard to the scale of a given project and its completion date. Table 84. Evaluation of influence of the RDP funds on execution of investment projects (gminas participating in the RDP) Question: If the gmina did not receive funds from the RDP: - Would a particular investment be executed? If the gmina did not receive funds from the RDP: - Would it result in a delayed execution of the investment? If the gmina did not receive funds from the RDP: - Would the investment be executed on a smaller scale? If the gmina did not receive funds from the RDP: - Would the investment be divided into stages? Answer: YES NO 52.0 48.0 74.0 26.0 18.0 82.0 58.0 42.0 Source: own survey. Considering the gminas which had not been supported financially by the RDP, in spite of the fact that they attempted to, 46 percent of them applied for funds from other sources; this was mainly a SAPARD program. The information in the table below shows the influence of the lack of funds from the RDP on execution of investment projects. 23 percent of the planned investment projects have not been executed at all. The indicator 296 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) describing the percentage of investment projects not executed, in our opinion shows the actual significance of the RDP funds, and at the same time indicates that the lack of financial support was a significant problem for gminas. Nearly 60 percent of the investment projects were delayed due to lack of financing from the RDP, and one in three investment projects has been executed on a smaller scale. Table 85. Evaluation of influence of the RDP funds on execution of investment projects (gminas not participating in the RDP) Answer: Question: Has the investment (planned under the RDP) - been executed? Has the fact that the gmina did not receive funds from the RDP - resulted in a delayed execution of a given investment? Has the fact that the gmina did not receive funds from the RDP - resulted in executing the investment on a smaller scale? Has the fact that the gmina did not receive funds from the RDP - resulted in dividing the investment into stages? YES NO 77.0 23.0 57.0 32.0 31.0 54.0 32.0 55.4 Source: own survey. In-depth interviews conducted with persons (the staff of gmina offices) in charge of the infrastructure investments execution imply that the funds received by gminas from the RDP, even in case of investment projects having financial “coverage” from other sources, played an important role in the infrastructure development in gminas, as the RDP funds gave an opportunity to carry out several investment projects concurrently, which definitely speeded up the process of infrastructure building. The Project was carried out in coordination with other programs (mainly with SAPARD, PHARE programs), and enhanced their positive impact on the rural infrastructure development process. In case of Component C the synergy effect has been accomplished. Table 86. Influence of external sources of funds on the process of rural infrastructure creation External sources of funds for financing rural infrastructure (for example funds from the EU, the World Bank): Yes No No opinion Speed up the infrastructure creation process 89.0 5.0 6.0 Equalize development opportunities of different gminas 78.0 11.0 11.0 Source: PBS 2005 survey results. In the opinion of local governments and business persons external sources of funds are a factor which speeds up the process of rural infrastructure creation and are also an element that equalizes development opportunities of different gminas. The significance of these funds is described as a key, decisive, governing factor (“without these funds one can not imagine doing three fourths of what can be done when these funds are available. Most investment projects would mean just repairing the most obvious deficiencies and gaps. This is the only chance to do bigger issues, bigger investment projects and a larger scope of issues to be done using local government funds, which was rather Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 297 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) unsuccessful”; “because it is the only opportunity for us to develop this infrastructure”; “...this road would be done but it is hard to tell exactly when, next year or some other time, and at the expense of another investment, and in this case the project has been completed...”; “when you can obtain funds from external sources, then such an investment is executed faster, and if it is faster, then also cheaper; as everyone knows that if an investment takes a long time, this unfortunately results in greater execution costs, therefore in this respect the external funds have a very positive influence on execution of such projects...”; “we took part in SAPARD and at present in IOPRD we will also execute an investment, and the RDP was a kind of a prelude to those bigger funds, and in my opinion the RDP was very necessary” — PBS 2005 survey). In gminas which submitted applications and did not receive funds, one can encounter, for example, such an opinion: “...I think that it would have been executed on a greater scale (“if the RDP funds had been available), and as it is now the version which we executed was a kind of an emergency version”). Also inhabitants of rural areas, the beneficiaries of different kinds of investment projects stated that the lack of funds was the main obstacle (80% respondents indicated that factor), and the poor rural community took the second place as the main barrier for infrastructure development (19% respondents indicated that factor). As the survey implies the role of funds from the RDP in infrastructure development is not questioned. However, the criteria for allocation of such funds raise doubts or maybe objections mainly in poorer, agricultural gminas. Preference for a greater own contribution is a “bonus” for wealthier gminas. Therefore it is not the funds, but the criteria in the opinion of local authorities of some gminas may lead to deepening of disproportions in the socio-economic level of development of gminas — “...the poorer regions should receive more of these funds. And this should be officially regulated, and not so that a person is able or not able to arrange something, because as it goes like this the difference will become even greater, because usually weaker gminas have weaker, less experienced people. A better gmina has better people, it pays more money and attracts better people...” PBS survey 2005. Survey results give sufficient grounds for concluding that the funds from the RDP have played an important role in the process of rural infrastructure creation. The financial support for infrastructure investment projects in rural areas in Poland provided by the RDP should be regarded as fully grounded. The RDP has operated jointly with other programs (mainly with SAPARD, PHARE programs), and has enhanced their positive influence on the process of rural infrastructure creation. From the point of view of its role in the process of infrastructure creation and the accomplished synergy effect the Component C should be given a positive assessment. Evaluation of adequacy of Component C with regard to the beneficiaries' needs The fact that local government officials indicate the lack of infrastructure as the main barrier for socio-economic development of gminas (66% of respondents indicated this) may be an evidence of the role and importance of infrastructure in local development. In the hierarchy of growth barriers for gminas, after insufficient presence of infrastructure elements, the lack of funds for investments is mentioned as the factor number two (both problems are directly interrelated). It should be pointed out that in the opinion of local governments infrastructure is perceived as a much bigger problem than unemployment (the percentage indicated was a half of that for infrastructure). This reflects the local governments' opinion (which we think is right) that infrastructure elements are the main 298 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) factor which is a prerequisite for improvement of economic situation of gminas, and hence an improvement in the labor market situation. An analysis of business persons' demand for infrastructure services was one of the elements of evaluation of the adequacy of activities carried out within Component C. The evaluation was conducted from the point of view of the importance of infrastructure in the development of business operations in rural areas and the role of individual infrastructure elements in conducting own business operations. Nearly 70% of business persons think that infrastructure elements have an influence on business operations, and over a half of this percentage judge this factor as very important. Only 15% of the surveyed business persons said that the infrastructure-related factor played no role in development of entrepreneurship. As far as the weight of individual infrastructure elements is concerned, in the business persons' opinion availability of a water supply system is the most important factor for conducting business operations. Based on the percentage of answers indicated by the respondents, good roads, and next telephone network and sewerage took next places. The marginal percentage of business persons who indicated organized solid waste disposal and gas system shows that those elements are not considered as a part of the necessary package of infrastructure elements. Table 87. Role of individual elements of infrastructure in conducting business operations (business persons' opinions) Investment type: Percentage indicated (priority) Water supply 35.0 Good roads 28.0 Telephone 18.0 Sewerage 16.0 Organized solid waste disposal 3.0 Source: PBS 2005 survey results. For representatives of rural business persons (owners of trade firms – 61%, service firms – 26%, manufacturing or construction firms) in case of particular business ventures conducted by them the priority of infrastructure elements is basically the same as above. The presence of a water supply system, good roads, sewerage and telephone network is regarded as necessary. Less than 50 percent of respondents consider solid waste disposal as a necessary condition for conducting business operations. Gas supply system is regarded, by a small percentage of respondents only as a useful element. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 299 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 88. Evaluation of importance of individual elements of infrastructure for conducting business operations For conducting business operations the particular element of infrastructure is: Type: Necessary Useful Water supply 70.0 26.0 Good roads 68.0 29.0 Sewerage 67.0 26.0 Telephone network 56.0 35.0 Solid waste disposal 48.0 45.5 Gas supply system 1.0 22.0 Source: PBS 2005 survey results. The low interest of local governments in investment projects concerning solid waste disposal corresponds to the low position of this element in the hierarchy of importance of individual elements from the point of view of their role in the development of business operations expressed by business persons. In the business persons' opinion the telephone network is an important element of infrastructure. Due to the fact that the needs here have been satisfied to a relatively high extent and low interest on the part of gminas, this infrastructure element was not covered by financial support under the RDP. In the rural business persons' opinion the most important infrastructure tasks to be executed include mainly roads construction, then sewerage, water supply system, and solid waste disposal sites as the last element. The hierarchy of individual infrastructure elements which require further development, proposed by the business persons matches the requests for individual types of investment projects submitted by gminas, which in our opinion reflect the biggest deficiencies in infrastructure in rural areas. Figure 54. Most important infrastructure projects to be executed in gminas (percentage indicated by business persons) 70% 60% 50% 40% 30% 20% 10% 0% roads construction sewerage water supply telephones waste disposal sites Source: PBS 2005 survey results. The RDP supported to the greatest extent those infrastructure activities which are of key importance from the point of view of business operations, and whose level of development is still unsatisfactory. 300 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Survey results give sufficient grounds for a positive assessment of adequacy of activities undertaken within Component C with regard to the needs of rural business persons. The financial support provided for those infrastructure elements was a factor contributing to the improvement of conditions for business operations in rural areas. Because different groups of beneficiaries could submit different requests for infrastructure services, the assessment of adequacy of the Project was also conducted from the point of view of its usefulness for individual groups, and first of all for inhabitants of rural areas. Among potential and actual beneficiaries of infrastructure services it is the households which constituted the most numerous group. The inhabitants of rural areas highly assessed the usefulness of particular infrastructure investment projects. Nearly 67% of the respondents, regardless of investment type considered the investment as being very useful, and 20% indicated that it was quite useful. If we exclude waste disposal sites, then practically among individual beneficiaries there was no one who would question the usefulness of the investment for a gmina. Considering individual investment types, if we take the percentage of “very useful” answers, the usefulness of water supply system development was given the highest mark, and the lowest mark was given to the organized waste disposal sites. In two cases out of four all beneficiaries expressed explicit positive opinion about usefulness of a given investment. Table 89. Evaluation of usefulness of infrastructure investment (village inhabitants) A particular investment was: Percentage indicated Very useful 67.0 Quite useful 20.0 Quite useless 2.5 Completely useless 3.7 Hard to tell 6.8 Source: PBS 2005 survey data. Considering the gminas' needs, the inhabitants regarded construction of waste disposal sites as least necessary. Only 34% of respondents said that such an investment was very useful, whereas according to 15% of the respondents the investment was not necessary at all. No other type of infrastructure had such a low level of social acceptance. With regard to this issue the opinions of inhabitants and rural business persons are similar. In our opinion this does not reflect a lack of awareness that waste disposal has to be an organized activity, but to a large extent this results from considerable needs in other spheres, which needs are more important from the point of view of improvement of living conditions for inhabitants of rural areas. Because the “roads” activity covered both construction and modernization of already existing roads, we have verified how needs are distributed between these two types of tasks. In the opinion of the inhabitants of rural areas, the most important needs in this sphere concern both construction and modernization of roads, however, the distribution of answers indicates that the needs in the sphere of modernization and repair are greater than the needs concerning construction of new road sections. The opinions of the Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 301 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) inhabitants correspond to the actual condition of rural roads in Poland, which first of all require an improvement of technical parameters. Table 90. Most important investment needs concerning rural roads What are the most important needs concerning roads in the gmina: Percentage indicated Construction of new road sections 18.2 Modernization and repair of existing roads 39.4 Construction of new roads and repair of existing roads 41.8 Construction of new roads is not necessary 0.3 Other, please specify 0.3 Total 100.0 Source: PBS 2005 survey results. Survey results give sufficient grounds for positive assessment of adequacy of the activities undertaken as part of Component C with regard to the needs of inhabitants of rural areas. The provision of financial support for these infrastructure elements was the most important task from the gminas' point of view. The fact that the financing covered not only construction of new roads but also modernization of already existing rural road sections should also be judged positively. However, we may wonder here if due to the very big interest in the three kinds of investments and a very small interest in waste disposal sites (this is confirmed by the relevant opinions of inhabitants and business persons) it was at all reasonable to undertake activity in this sphere and whether concentration of funds and efforts on the three priority activities would be a better option. Training The objective of the activity “Training and technical support” executed as part of Component C in line with the assumptions of the Project was to create alternative organizational and institutional structures with a view to ensure stable and available municipal services and introduce market orientation into the process of infrastructure construction and operation. The training and technical support were intended to be useful in projection of infrastructure investments. The adequacy of this activity has been assessed on the basis of the beneficiaries' opinion about its usefulness in the phase of projection of infrastructure investment, optimum with regard to financial, economic, social and application preparation aspects. As far as the evaluation of the training is concerned, local government officials had a very high opinion of the usefulness of the training conducted as part of Component C. The figure below shows that no participant assessed the training as quite useless or completely useless. However, it should be emphasized that 30% of the surveyed gminas declared that they did not participate in any training as part of Component C (answers: I don't know, I can't remember or no answer given). As we know the training and technical support covered all potential beneficiaries. The relatively high share of gminas which did not declare that they had participated most probably is due to a very large number of various training courses which are offered to local governments, and also execution of investment projects by means of funds coming from various sources. Looking back in time, it is difficult at present to assign the conducted courses of training to particular projects. 302 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Figure 55. Evaluation of usefulness of training during execution of investment projects 60 50 40 % 30 20 10 0 Quite useful Very useful Hard to tell Source: PBS 2005 survey results. Local government officials who said during the survey that they participated in the training had a very high opinion of the quality and usefulness of the training (“The program was very well prepared with regard to the training aspect. It seems to me that the training can be awarded a “very good” mark. I think that those who wanted, could understand the issues and will be able to implement this...”; “...very well explained procedure and first of all the keeping of all that documentation and filling in those documents, I was surprised, because I take part in many courses of training, and so far I haven't participated in any other training so well and efficiently conducted as this one. It was very well explained, indeed.” – PBS 2005 survey). Only 27% of the surveyed gminas confirmed receiving technical support during execution of an infrastructure investment within Component C. For further 22% of gminas answers have not been given. In case of a part of local governments which declared in the survey that they received technical support, 65% of the respondents said that it was very useful, 35% said that it was useful, so there were no objections concerning the quality of the technical support. The high percentage of answers indicating that a gmina did not receive support is due to similar reasons as in case of training under Component C. Some of the surveyed gminas declared that they did not use the support because they did not need it. The usefulness of the “training and technical support” activity should be considered by measuring to what extent the proposed solutions concerning design of infrastructure investment projects were accepted and implemented by local government units applying for the RDP funds. The proposed economic model for investment projection has been used by 34% of the surveyed gminas, 24% of gminas declared that they did not use it while planning the investment projects, and the remaining cases mean a lack of answer or no opinion given. Out of the 34% gminas which declared that they used such a model, only 6 surveyed officials assessed it as useless. As the needs in the sphere of infrastructure development are enormous and at the same time the financial capacity of local governments is limited, the issue of selecting an optimum infrastructure investment to be executed, from the financial, economic and social point of view becomes a matter of particular importance, and the surveyed local government officials agree with this opinion. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 303 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The adequacy of the training with regard to the needs of the beneficiaries (gminas) should be judged very positively, and also the quality of the training should be given a high mark. Evaluation of effectiveness of Component C according to the beneficiaries The local governments have a relatively high opinion of the scope of executed goals in comparison with the planned ones with respect to the RDP infrastructure investment projects. In 63% of the surveyed gminas local governments considered that they had managed to accomplish all planned goals, 32% considered that the planned goals had been partly accomplished. The percentage of gminas which did not accomplish what they had planned was marginal and amounted to 2 percent only. Taking individual activities into consideration, investments in road network are characterized by highest effectiveness, and investments in sewerage are characterized by lowest effectiveness. A completed investment is regarded by a local government as a local success and an incentive to get involved in further activities. Table 91. Evaluation of planned investment goals (local governments) Have all planned goals been accomplished: Percentage indicated Yes, all planned goals have been accomplished. 63.0 Yes, the planned goals have been partly accomplished. 32.0 No 2.0 It's hard to tell. 2.0 TOTAL 100.0 Source: PBS 2005 survey results. The greatest discrepancy between the planned and accomplished results concerns the number of created jobs and the final number of service recipients. In local government officials' opinion the local nature of an investment, and thus its small scale is the main reason explaining why it has not produced greater business activity, and also the short time from completion of the investment (local government officials in gminas in which there were differences between the planned and actual number of jobs knew the rules for final settlement of the task well and they referred to this). Another factor indicated by the local government is the bad economic situation of rural population which has been translated into a smaller than planned number of households using a particular infrastructure service. In the survey conducted in households which have become connected to a water supply system or a sewerage system as part of the RDP, 34 percent of respondents answered “yes” in both cases to the question “Were there any persons at a given locality who had the opportunity to become connected to a water supply and sewerage system, but have not become connected?” Also in case of waste disposal sites the respondents indicated that a similar situation occurred. Situations revealed in other surveys, in which situations economic circumstances are a barrier for access to infrastructure services for some social groups have thus been again confirmed. 304 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) It should be emphasized that in a number of cases where differences between planned and actual values occurred, it was not possible to clarify the matter fully in the gminas during the survey. However, while evaluating the effectiveness of the Project it is important to find out the reasons for such differences. The surveys have confirmed that in spite of the physical possibility some inhabitants (for economic reasons) decide not to use all or a part of the newly available services. One of the explanations of the difference between the planned and actual water consumption is the smaller than planned number of end users, and also (when the planned number of end users has been accomplished) water consumption being smaller than planned. Nearly 60 percent of surveyed households try to reduce their water consumption, and a half of them does so mainly due to the high price of water. Some inhabitants declared that in spite of having been connected to the water supply system they still use water from a local system (hydrophore) or from a well, mainly for farming-related purposes. In a few cases the respondents (connected to a water supply system as part of the RDP) said that the amount exceeded ten buckets of water per day. This amount of water might be sold potentially. The survey shows that the smaller the water consumption, the higher the unit costs, and lower investment efficiency. Therefore a projection of infrastructure investment projects based on actual parameters concerning the number of potential end users and the volume of consumed water is an important issue. According to local government officials an opinion (which is right) prevails that “the charges should be calculated so that they cover the actual costs incurred by the producer. Whether the costs of water or sewage treatment per cubic meter allow the company to continue operations. And at the same time they should not be an excessive burden for inhabitants. Normally this is done in such a way that calculations are made and charges are set accordingly” (PBS 2005 survey). The information above shows that the goal of Component C, assumed in the Project, concerning introduction of market orientation into the process of construction and operation of rural infrastructure was based on a correct assumption that low cost of infrastructure services encourages potential beneficiaries to use these facilities (PAD p. 14). These assumptions show that there is a need for a projection of infrastructure investments, which are adequate to the scale of needs, and also ensure provision of services of appropriate quality and at prices which match financial possibilities of potential users. The survey shows that this is an important aspect of rural infrastructure development, which is accepted by local governments, although the survey results show that it has not been executed completely. The part of potential end users, which have not used a given investment at all and a lower than planned water consumption and volume of discharged sewage may serve as a measure of this lack of efficiency. Evaluation of usefulness of the Project according to the beneficiaries The evaluation of the usefulness of the Project included an analysis of the influence of executed infrastructure investment projects on: • growth of business activity, • improvement of conditions for business operations, • improvement of living conditions of rural population, • improvement of health of rural population. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 305 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Influence of infrastructure investment projects on growth of business activity and improvement of conditions for business operations The investment analysis was conducted from the point of view of the role of a given investment as a factor stimulating business activity and its significance in improvement of conditions for business firm operations. Generally, the percentage of firms which have been set up as a result of (or arising from) the launched investment was definitely lower than the percentage of firms for which setting up of the firm was indifferent with regard to a given investment. On average 20 percent of business persons declared a relationship between coming into existence in a local economic structure and a given infrastructure investment. There is a significant difference among individual investment types with respect to this. Investment projects connected with development of water supply network had the greatest influence on setting up of business entities, and the projects connected with construction or modernization of roads had the smallest influence. On average 41 percent of business persons confirmed that a particular infrastructure investment had had an impact on operations of a given firm; the water supply system had the greatest significance, similarly as in case above. Table 92. Evaluation of influence of infrastructure investment on establishing and operating of a firm The investment concerning the water supply system had influence on: YES NO Establishing of the firm 31.0 69.0 Operations of the firm 55.0 45.0 The investment concerning the sewerage had influence on: YES NO Establishing of the firm 25.0 75.0 Operations of the firm 34.0 66.0 The investment concerning construction/modernization of a road had influence on: YES NO Establishing of the firm 4.0 96.0 Operations of the firm 35.0 65.0 Source: PBS 2005 survey results. The influence of investment projects on business operations was also analyzed by means of an indicator specifying the percentage of surveyed firms which provided or produced for the benefit of a given infrastructure investment. 13 percent of surveyed firms declared a direct or indirect relation with a newly executed investment. Nearly 10 percent of business persons indicated that the infrastructure investment contributed to an improvement of the financial standing of their firm and to creation of new income sources for the firm. 306 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The percentage of business persons who purchased machines (equipment) for a given enterprise after the launch of a given infrastructure investment (created with the RDP funds assistance) in case of individual investment types was as follows: • water supply – 24%, • roads – 0.0%, • sewerage - 16%. Practically in one in five surveyed firms changes in standard of equipment took place during execution or after completion of a water supply or sewerage investment. Investments in a water supply system had the strongest positive influence on improvement of a firm's equipment, whereas in case of modernization of a road or construction of a new road such a relationship has not been recorded. The usefulness of the Project has also been analyzed from the point of view of the influence of infrastructure investment projects on changes in production scale and employment in surveyed firms. The analyses of empirical material show that the Project has resulted in positive changes (to a different extent) in this respect. The highest percentage of firms indicated that the infrastructure investment projects helped them survive in the market. This result to some extent reflects the economic standing of rural firms. For some firms the key problem is not employment growth but simply to survive, i.e. to continue existence in the market. If thanks to the Project 28 percent of surveyed firms managed to survive a difficult period, then such a result allows to assess it positively in this context. An increase in scale of production was recorded in 16 percent of gminas, and an increased scope of a firm's influence was declared by 14% of surveyed respondents. Considering the employment growth which was noted as a result of (due to execution and launch) a given investment, 9 percent of surveyed firms have recorded changes with respect to this. In case of 17 percent of firms the execution of a given infrastructure investment was the determining factor when deciding to locate in a locality where the investment had been executed. Table 93. Influence of infrastructure investment on the scope of operations of firms The executed infrastructure investment made the following contribution: YES (Percentage indicated) Made it possible to increase the scale of production of the firm 16.0 Made it possible to increase the scope of influence of the firm 14.0 Employment growth 9.0 Made it possible to survive in the market 28.0 Was the key factor when deciding to locate the firm at a given place 17.0 Source: PBS 2005 survey data. In spite of the fact that in business persons' opinion good roads are a key factor in growth of business operations in rural areas (business persons point out this element as one of the main problems to be solved in local systems), however, its role in the influence on the Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 307 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) scope and scale of the firm, employment growth is smallest among the elements covered by the RDP financing. We have also analyzed the usefulness of the Project from the point of view of the influence of executed facilities on the costs of conducting business operations. The share of firms for which the executed investment resulted in decreased costs of conducting business operations was four times higher than the share of firms for which the executed investment resulted in increased costs. However, it is the business firms for which the executed investment had no influence on a change of costs of conducting business operations which prevailed (in different scope depending on the type of activity). Table 94. Influence of infrastructure investment on the costs of conducting business operations Has the investment resulted in: Investment type: Decreased costs Increased costs Had no influence on costs Water supply 17.0 7.0 76.0 Roads 0.0 2.1 97.2 Sewerage 16.0 2.3 82.0 Source: PBS 2005 survey results. The percentage of firm owners for whom the costs incurred for provided infrastructure investment projects were too high was marginal. The executed investment projects more often generated a decrease in financial burden than an increase. As far as the influence of infrastructure investment projects on the general economic situation of a gmina is concerned, from the point of view of local governments such an influence is noticed to a different extent. In case of suburban gminas, after connection to a water supply or sewerage system the interest in building lots in rural areas, expressed mainly by urban population has increased. After infrastructure has been created in a given area, also interest in recreation lots has increased. In 14 percent of the surveyed gminas the local governments declared that after execution of an infrastructure investment new sources of income have appeared. In areas where a particular infrastructure investment is located local government officials record new (single) business entities, also the number of accommodation places increases. However, the respondents express an opinion here that they find it hard to tell to what extent this is due to the influence of a given investment or the influence of other conditions. As far as the infrastructure building process is concerned, its influence on economic development of rural environments was definitely lower than expected. On the basis of interviews with local government officials we can conclude that in most of cases the investment-related works were executed by external firms, due to their better financial standing. However, some signals from local areas confirmed that in terms of reference for bidders there were provisions concerning a necessity to offer employment in works connected with infrastructure creation to unemployed persons in a given gmina, most often in manual jobs (the survey results have provided particular examples confirming that in some cases this resulted in a permanent employment offer directed to such an unemployed person). Also in the farming sector the positive influence of these investments can be seen, which may be confirmed by opinions of local government officials about the influence of the 308 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) investment projects on business activity growth — “the activity of inhabitants has increased, because what we can see now, this connection to a water system, and this coincides with Poland's accession to the EU, people began to develop farm buildings, they increase the number of breeding animals, so this has a positive influence”; “undoubtedly it had, because even those opportunists who at the time of the investment execution did not want water, at present make endeavours to have water supplied to them. First of all real life situations proved this. After Poland's accession to the EU, the EU imposed strict requirements concerning water. Water which is given to cattle has to have relevant certificates and satisfy relevant standards. And water from dug wells, which was used by people there, did not meet such requirements. Therefore certainly this has resulted in higher living standards and standards in farms. Not only humans, but also animals. Because people can hand over the fruit of their labor and sell further. There is no problem”... PBS 2005 survey). The field studies have provided many such specific positive examples of the influence of an infrastructure investment on different aspects of economic and social life. Due to their nature we can not make general conclusions based on the “case studies”, nevertheless they show that some processes have been initiated as a result of the investment projects and that these projects have (may have) a wide spectrum of influence in different types of rural areas and different gminas. It is very difficult to assess the real effects of the influence of infrastructure investment projects, for example due to the fact that the influence of an investment may be spread over a long period of time. The usefulness of the Project from the point of view of improvement of conditions for operation of firms and stimulation of economic activity should be assessed as positive. Considering the specific selection of firms, the relatively low number of firms, the percentage of business entities which creation and operation was influenced by the executed investment should be regarded as high. The executed investment projects (with the exception of roads) influenced all surveyed aspects of a firm's operation determining its growth (increased output, employment growth, increased scope of influence, better equipment, lowered costs of business operations). The infrastructure investment projects have also resulted in greater investment and tourist attractiveness of some areas (this was manifested directly by increased demand for building and recreation lots), which in a longer perspective may result in greater demand for building materials and enhanced economic activity of local labor force. Influence of infrastructure investment projects on improvement of living conditions and health of rural population In the opinion of both local government officials and inhabitants (beneficiaries) the infrastructure investment projects executed as part of the RDP had the greatest influence on improvement of living conditions of the rural population. Nearly 90 percent of local government officials of gminas participating in the RDP confirmed positive influence of infrastructure investment projects on living conditions of the rural population. 60 percent of rural beneficiaries indicated in the first place the living conditions as the sphere where the influence of the introduced facilities was strongest. The percentage of respondents who in the first place indicated the improved appearance of villages as a positive result of the new infrastructure investment (22 percent of inhabitants) was relatively high. 13 percent of respondents indicated improved health situation as the most important element. It should be emphasized here that in the inhabitants' opinion the strength of influence on living conditions and health situation was much higher than on improvement of conditions of business operations and economic activity growth. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 309 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) As far as the assessment of the Project's influence on improvement of financial situation of rural families is concerned, on the basis of the results of questionnaire surveys we have not noticed any relationship of this kind. Practically only one respondent declared that thanks to the investment he/she had gained new sources of income. The main problem for households not having access to water mains is no possibility to use various domestic appliances, and poor quality of water taken from other systems, particularly from wells is mentioned in the second place (“...the fact that people have water, they are glad because they don't have to worry about sanitary and epidemiological inspection. They don't need to go with samples from their wells to the sanitary and epidemiological inspection and pay for them quite significant amounts of money. They take water from the municipal water supply system, where water is of good quality” – PBS 2005 survey). After connection to a water supply system in 35 percent of surveyed households more appliances were introduced which could not be used earlier. In 31 percent of households the water supply system investment was an impulse for construction and repair of part of a house (this mainly concerns the bathroom and the kitchen). The majority of work in this sphere was carried out without using third-party services. However, in a relatively small number of cases third party contractors were employed. In case of a sewerage investment 18 percent of households participating in this activity executed construction or repair in the house (this concerned the bathroom or the kitchen), of which in 6 cases with involvement of employed labor force. Of course this was a seasonal employment, but considering the existing great problems in the local market even ad hoc work can generate measurable benefits. The survey shows that the Project to a different extent initiated initiatives which may result in an improvement of the economic situation, and also initiated multiplier effects (for example increased demand for building materials or repair-related materials). This indirect influence due to its specific nature is very difficult to diagnose and it is even more difficult to specify its actual size. The influence of road investment projects on improvement of living conditions was assessed as positive by inhabitants of rural areas (taking as the basis the relation between those who said that the conditions had improved and those who had an opposite opinion). The percentage of those who said that the investment produced changes was at a level similar to the percentage of those who said that the road investment had not generated any effect. 310 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 95. Influence of road investment projects on living conditions of rural population When the road investment was completed the living conditions of the family: Percentage indicated Have definitely improved 9.2 Have generally improved 30.3 Have not changed 40.8 Have generally worsened 3.9 It is hard to tell 15.8 TOTAL 100.0 Source: PBS 2005 survey results. In the opinion of inhabitants of rural areas the road investment projects have not generally resulted in a smaller number of road accidents. Nearly 70 percent of respondents said that the investment had no influence on the number of collisions. The percentage of respondents who said that the number of accidents decreased was minimum and amounted to 5 percent and was equal to approx. a half of the percentage of respondents who said that the investment resulted in worsening of the situation in this sphere. The distribution of judgements concerning the influence of a road investment on changes in the number of car damages is very similar. Also in this respect the gathered information shows that a positive influence of infrastructure investment on a decrease in the number of car damages has not been observed. Nearly 7 percent of respondents said that the road investment has produced changes in this respect. Table 96. Influence of road investment on the number of road accidents and car damages When the road investment was completed: Number of road accidents in gmina Number of car damages Percentage indicated Has increased 9.7 8.0 Has not changed 68.0 67.0 Has decreased 5.0 6.6 No opinion 17.0 18.4 Total 100.0 100.0 Source: PBS 2005 survey results. The influence of the road investment on improvement of access within a gmina and outside a gmina was assessed much more positively. No respondent said that after a change in roads standard the situation had become worse. The group of persons who were of the opinion that an improvement of roads standard has not resulted in changes with regard to access dominated; however 43 percent of beneficiaries noticed an improvement in access to a locality in their gmina. The analysis shows that after completion of the investment access to school for children improved most, and access to a town nearby improved least. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 311 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 97. Influence of road investment on better spatial access After completion of the road investment, access: to a locality in gmina to a town nearby between villages to school for children has improved 43.0 21.0 26.0 60.0 has not changed 47.0 70.0 60.0 18.0 has become worse 0.0 0.0 0.0 0.0 no opinion 10.0 9.0 14.0 22.0 Total 100.0 100.0 100.0 100.0 Source: PBS 2005 survey results. 18 percent of respondents reported improved access to fields as a result of the investment, and 37 percent of respondents reported greater safety (especially access for an ambulance). The very small number of answers concerning the influence of a road investment on road traffic intensity does not allow to evaluate this aspect. Only in 3 gminas the provided information was based on measurements, whereas in other gminas they were based only on estimates. In some gminas local government officials explained that they did not carry out such analyses, because, as they stated “they do not have time for this”. It should be stated once more here that for evaluation of the Project it is necessary to select appropriate indicators which are registered day by day, and systematic verification and updating of the indicators should be a duty of the gminas participating in the Project. After completion of an investment a gmina firstly is not interested, and secondly not obligated to provide this kind of information. However, this information can be useful while constructing other projects, and especially while setting criteria for funds allocation and methods of settlement of the generated effects. The usefulness of the Project from the point of view of influence of infrastructure investment projects on improvement of living and health conditions should be assessed as positive. The investments in water supply systems had the greatest influence (broadest scope of effects) on improvement of living conditions. Evaluation of burden arising from installation and operation costs The empirical research has confirmed that installation and operation costs are for some social groups a barrier for access to infrastructure services. In local government officials' opinion unemployed persons are in the least privileged position (77% indicated this), and farmers come next (27% indicated this). Based on the gathered data the cost of installation of a water supply system or sewerage was reported as a burden by nearly 60% of respondents (recipients of infrastructure services created as part of the RDP). 26% of respondents (connected to a water supply system or sewerage) said that the cost of the installation was very high for their budgets. Practically only one in five families did not regard the cost connected with installation of a water supply system or sewerage as a financial burden. There are no significant differences in opinions about this matter between recipients of services created as part of the RDP and those created with assistance from other projects. 312 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 98. Evaluation of burden arising from installation of a water supply system and sewerage (inhabitants' opinions) Burden arising from costs of installation of a water supply system or sewerage: Percentage indicated Very high 26.0 Generally high 33.0 Generally low 20.0 Very low 3.0 No burden at all for us 18.0 TOTAL 100.0 Source: PBS 2005 survey results. The distribution of answers to the question below shows that charges for water and sewage are a significant burden for the households. This problem affects communities which use a water supply system or sewerage built as part of the RDP and also those using services of systems created in the same period, but with assistance from other projects. In gminas participating in the RDP exactly a half of the surveyed local government officials said that the cost of water and sewage is a significant burden for rural population, whereas in gminas not participating in the RDP the percentage of respondents confirming such a situation was higher by 18 percentage points. For 44 percent of households water bills are a significant burden, whereas for 65 percent of households the costs of water bills and sewage bills are regarded as high. 62 percent of surveyed inhabitants of rural areas think that the prices of water and sewage are too high with regard to the economic situation of the rural population. Table 99. Evaluation of charges for water and sewage (inhabitants' opinions) Are charges for water and sewage a significant financial burden for inhabitants: Answer: (% indicated) YES NO I DON'T KNOW Gminas participating in the RDP 50.0 37.5 12.5 Gminas not participating in the RDP 68.0 28.0 4.0 Source: PBS 2005 survey results. According to the assumptions of the Project the charges for infrastructure services created as part of the RDP should not exceed 3-5% of total disposable income of a household. The empirical research shows that these assumptions of the Project have practically been accomplished. The average income of a family in a gmina where an infrastructure investment was executed amounted to PLN 1428, which means that according to the assumptions of the Project the charge should not exceed PLN 71. Based on the analysis of gathered data in 97% of the surveyed households the monthly charge for water does not exceed this amount. Only in 3% of cases the burden arising from this source was greater than planned. Based on the table below for 63% of surveyed households the charges for water do not exceed on average PLN 50 per month. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 313 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 100. Monthly charges for water* Monthly charge for water Percentage indicated up to PLN 10 18.2 PLN 11 – 30 63.3 PLN 31 – 50 11.5 PLN 51 and more 7.0 of which more than PLN 71 3.0 Total 100.0 * only those households that gave an answer have been taken into account Source: PBS 2005 survey results (only the households which gave answers are included). Although average burden arising from charges for public utility services does not generally exceed the assumed threshold, sometimes it happens that inhabitants of rural areas do not pay in due time. 10% of surveyed households said that they had problems to make water bill payments on time, and in case of water bills and sewage bills 30 % said so. Based on the results of the survey covering local government officials only in one of three surveyed gminas regular collection of receivables for public utility services is not a problem. Table 101. Opinions concerning main criterion for setting prices of public utility services Main criterion for setting prices of public utility services: Gminas not Gminas participating in participating in the RDP the RDP Economic aspects 53.0 52.1 Social aspects 41.0 33.3 Both economic and social aspects 3.1 10.4 It's hard to tell 3.1 4.2 100.0 100.0 Total Source: PBS 2005 survey results. Both in gminas participating in the RDP and gminas not participating in the Project the local governments practically to the same degree indicate economic aspects while setting criteria for charges for public utility services. The percentage of supporters of economic criteria, and thus at the same time market solutions in the process of operation of municipal infrastructure in both cases is greater than 50%. However, the percentage of those who think that while setting charges for services one should first of all consider social aspects is quite significant. Some local government officials think that one should subsidize infrastructure investment projects due to firstly bad economic situation of rural population (an element of equal opportunities policy) and secondly due to economic benefits which can be obtained from such a solution in longer perspective. The argument that the price of discharged sewage should be subsidized (for a certain period of time) so as to prevent uncontrolled discharge of sewage to rivers or fields (which is a real problem if development of water supply system is fast and significantly precedes development of sewerage) is convincing. From the point of view of “repair” of the natural environment as a result of such actions, 314 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) subsidizing public utility services may prove to be a less expensive solution in a longer time perspective. To sum up, both installation and operation costs of infrastructure services are a significant burden and act as a barrier for access in case of some social groups. Therefore assuming a projection of investments in the Project in accordance with the needs and possibilities of rural population and introducing special “training and technical support” activities so as to implement this should be regarded as reasonable. 5.3 Summary 5.3.1 Analysis of Component C benefits As a result of realizing Component C a number of investment projects have been executed which have contributed to an improvement of situation with regard to infrastructure facilities in the areas covered by the Project. The measurable effects of Component C execution include: • 923 km of water supply systems, • 14 water treatment plants (three new plants, and the remaining ones have been modernized), • 1037 km of sewerage, • 236 sewage treatment plants (of which 213 in farms), • 4 solid waste disposal sites, • 734 km of roads (modernized and built). As a result of execution of these investment projects: • the number of users connected to a water supply system has increased by 13.3 thousand, • the number of users connected to sewerage has increased by 22.3 thousand, • the number of persons using organized waste disposal sites has increased by 23.1 thousand. The scale of demand for infrastructure investment projects measured by the number of submitted applications and their bill of quantities value (five times higher than the value of subsidies), and also the high level of demand for such services in other projects (for example SAPARD) show that generally without the Project's contribution the length of water supply network and sewerage, and the number of: service connections, sewage treatment plants, waste disposal sites, built and modernized roads would be lower by the measurable effects generated by the Project. The products created within Component C can be regarded as net effect of the Project. The countable benefits obtained as a result of Component C execution include also new jobs. The effects above are translated to economic benefits such as a reduced amount of Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 315 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) unemployment benefits paid out, and increased state budget income arising from income tax and turnover tax. As a result of infrastructure investments execution 5940 new jobs have been created in rural areas. The created jobs were to a different degree generated by different activities. Based on the chart below the “roads construction and modernization” activity had the greatest share with regard to this. 3704 jobs (62.4%) have been created as part of the RDP as a result of development of the road network. Figure 56. Breakdown of new jobs created in Component C by activity water supply sewerage roads solid waste Source: own analysis based on PCU report. The method applied in the Chapter IV.6 – “Analysis of socio-economic effects of the RDP” has been used for evaluation of benefits for the State budget arising from creation of jobs. In accordance with calculations presented in the first part of this chapter, the value of budget benefits arising from creation of one job for an unemployed person, for the 2003 year data, was between PLN 4 582 and PLN 11 852 per year, depending on the remuneration of the unemployed person. Assuming that creation of new jobs results in a corresponding decrease in the number of unemployed persons we obtain the level of State budget benefits arising from creation of 5940 jobs as a result of execution of Component C of the RDP. This benefit was between PLN 27.21 million and PLN 70.40 million, depending on the average remuneration received by an unemployed person taking a job. Among the benefits of infrastructure investments contributing to a greater attractiveness (in broad sense) of a gmina we can include greater tax income for the gmina (if there is an inflow of people, who are often more affluent), tax charges arising from purchase of building or recreation lots. These are measurable benefits for a gmina's budget, which are recorded by local government officials in their gminas. However, it is very difficult to estimate them, especially because the indirect positive influence of infrastructure investment projects may be progressive as time goes by. This influence also generates indirect effects – greater building activity means greater demand for local labor force and building materials. Thus further income for the budget is generated. The opinions above are not imaginary but are based on results of empirical research providing specific examples showing that the process of positive influence and multiplier effects (in different places and with varied intensity) has been initiated. The measurable effects arise not only from construction but also from modernization of infrastructure equipment and facilities, which is often connected with a reduction in 316 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) energy consumption and losses level (local governments' opinion), which is translated to a lower price of a given service and better collection of receivables. The activities within Component C have also produced many effects which can not be expressed in monetary units. The conducted statistical analysis shows that the Project has contributed to a decrease in disproportion between the number of water mains service connections and sewerage service connections. This problem has very unfavourable consequences from the point of view of health of rural population and quality of the environment (which also is directly connected with the former element). Due to the contribution (number of water mains service connections) of the Project the disproportion in this sphere has been reduced by 10% for the whole country, and by 30% for gminas participating in the RDP. This fact is especially worth emphasizing, particularly in the situation of a deepening disproportion between the number of water mains service connections and sewerage service connections. As a result of this situation the natural environment and health of the rural population are increasingly threatened. It is not possible to express the actual effect accomplished in this sphere in monetary units. The weight of the effect is big and we assume that the positive effect of these changes will last for a long time. The survey implies that execution of infrastructure investment projects has contributed both to an improvement in living and work conditions (also in farms) and conditions for business. From the point of view of improvement of living conditions of rural population the most important accomplishments include: • conditions facilitating work in a household (a significant improvement in the standard/quantity of durable goods in households has been recorded); • improved technical condition of dwellings (repair, modernization); • better access (first of all clearly indicated improvement of transport for children to school); • improved appearance of villages. As a result of execution of Component C different activities to a different extent positively influenced the conditions for conducting business operations. The survey also confirms that the infrastructure investment projects had an influence on growth of business activity of the rural population which has been reflected for example by increased number of accommodation places in agritourism farms. The beneficiaries in interviews emphasized the role of the Project as a factor which stimulates local governments to undertake activity and also as a factor conducive to greater activity of the rural population. 5.3.2 Long term influence of Component C The process of rural infrastructure creation in Poland has a specific nature arising from considerable backwardness and great investment needs, and also from a favourable situation as far as external sources of funds are concerned. As a result of these circumstances the process of infrastructure development (and also modernization) is taking place at a fast pace, which improves the complementarity and complexity of infrastructure in gminas. This allows us to make a conclusion that the effect of the Project in this Component will be long-lasting and in longer perspective should generate greater external effects than at present. Based on the results of field investigations and also an analysis of spatial distribution of participants (gminas), for example in SAPARD, we can Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 317 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) expect that the scale of multiplier effects for the investment projects and synergy effects with other projects will be growing. It should be added that 93% of gminas participating in the RDP in Component C at the same time participated in SAPARD. The results of the survey covering local government officials may convince us to share the opinion that the effect of Component C will last for a long time. The table below shows that the percentage of those surveyed local government officials who by participation in the RDP were encouraged to further investment activity was dominant. Table 102. Influence of participation in the RDP on subsequent investment activities Has preparation of the application and the project under the RDP: Percentage indicated encouraged you to continue investment activities 81.6 discouraged you to continue investment activities 1.3 neither encouraged nor discouraged 17.1 TOTAL 100.0 Source: Own surveys. The positive effects of the Project which will be noticeable over longer time include changed attitude of beneficiaries (inhabitants) to execution of next investment projects and changes in living conditions standards. The positive effect, which in future will be translated to many new elements, is certainly a greater awareness of inhabitants and local governments, which was emphasized by the respondents with respect to the necessity to pay the costs of infrastructure services, pay the costs connected with improved living conditions and improved quality of natural environment, and that these aspects are important not only from the point of view of living conditions but also from the point of view of economic growth opportunities for the whole gmina. 5.3.3 Activities executed as part of Component C The execution of the Component began with setting the dates for submitting applications and dates for training of staff of institutions implementing the Project in the sphere of filling in the applications (end of the year 2000). The RSCs relatively quickly drew up ranking lists (from March 2001 to June 2001). Depending on voivodship the lists were drawn up for the whole Component or for individual types of infrastructure. At the same time a model agreement with beneficiaries concerning financing was drawn up (April 2001) and training concerning preparation of applications were conducted. Due to the considerable interest in Component C expressed by VPIUs, local offices of ARMA, gminas and powiats the PCU provided timely detailed answers to questions concerning the rules for Component C execution (the answers to the questions were placed in web pages, information concerning implementation of Component C was also published in the national local governments weekly “Wspólnota”.) 318 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Figure 57. Basic tasks of Component C – implementation timing Source: Consultant’s own analyses based on Interim Reports of Program Management, PCU The process of submitting applications for funds from the RDP designed for infrastructure investments ended in the second quarter of 2001. The local governments submitted 2077 applications, of which 14.1% were verified negatively (most often due to not delivering required enclosures in due time), and another 4.9% (positively verified) were rejected due to for example obtained financing from other projects. Finally 476 applications (which constituted 28.3% of the total number of applications) were approved for financing. The value of submitted applications exceeded five times the amount envisaged for Component C execution. The chart below shows that road-related applications prevailed among both submitted and approved applications. Figure 58. Submitted and approved applications breakdown Road S e w e ra g e W ate r s . S o lid w a s te dis p os al 100 % 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% S u bm itte d 2 0 0 1 A p p ro ve d 2 0 0 1 A c tu ally e xe c u te d 2 0 0 4 Source: PCU quarterly reports. The first agreements for financing were signed in the second quarter of 2001. The first contracts with investment contractors were concluded in July, and the works began in August 2001. The execution of first investment projects began in September 2001. By the end of the year invoices amounting to 36.9% of the value of investment projects under the signed contracts were submitted. As of the end of the year 2001 127 investment Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 319 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) projects were completed (of which road projects accounted for 81%, and water supply system projects accounted for 12.6%). In 2002 further progress in investment execution was recorded. Next 301 infrastructure investment projects were completed (of which road projects accounted for nearly 59%, sewerage projects accounted for 24.3% and water supply system projects accounted for 16.8%). The year 2002 was the year in which the intensification of activities measured by the quantity and percentage of completed projects was the highest (nearly 42%). Due to the accumulated volume of works and consequently the big number of applications to repay invoices submitted at the end of 2001, in the first quarter of 2002 there had been delays in transfer of subsidies from the RDP to gminas and powiats. The ministry of Agriculture and Rural Development together with the PCU made a considerable endeavour to improve the process in next months. The signing of financing agreements and concluding of investment contracts was continued in 2003 and 2004. The distribution of concluded and executed contracts in time is presented in the table below. Table 103. The number of signed and completed contracts in the years 20012004 As of year end: Signed contracts Completed contracts Number Percentage Number Percentage 2001 536 74.1 127 17.6 2002 585 80.9 428 59.2 2003 691 95.6 666 92.1 2004 723 100.0 693 95.9 Source: Own calculations based on PCU quarterly reports. Based on the table above the number of finally signed contracts in comparison with the number for the second quarter of 2001 increased considerably. Already in the fourth quarter of 2001 the first savings resulting from the first bidding procedures allowed to execute projects which were ranked lower in the ranking lists; this process continued throughout the whole Project. As a result of obtained savings after the settled subsequent bidding procedures the number of projects approved by RSC for financing under the RDP increased until the end of 2004 by 247 (i.e. by 52% in comparison with the initial number, which was 476 in the second quarter of 2001). In the first quarter of 2003 the zachodniopomorskie voivodship made a decision to allow submitting new applications for financing investment projects in Component C, because all items in the ranking list from the previous selection procedure had already been used. In 2003 the execution of Component C had virtually been completed. What was still to be made were only final payments and investment projects being at their final stages, close to completion. The execution of next 238 investment projects was completed. Out of the total number of 691 signed contracts, 666 investment projects (i.e. 96.4% of all planned) were completed as of the end of 2003, of which road projects accounted for nearly 57%, sewerage projects accounted for 28.4%, and water supply system projects accounted for 14.1%. The last contracts were signed in September 2004. The unused funds were moved by many voivodships to investment projects in Sub-Component B2. In 2004 the physical execution of investment projects in Component C was completed. The settlement and closing of all contracts is planned to take place until March 2005. In Component C in total 723 contracts totaling approx. PLN 471 million were concluded until the end of December 2004, and execution of 693 contracts was completed. 320 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Table 104. Contracts in Component C by investment type and execution period Roads Sewerage Water supply Solid waste As of year end I II I II I II I II 2001 68.5 24.1 80.4 3.1 84.5 17.6 100.0 50.0 2002 74.1 57.9 90.7 53.6 90.7 74.2 100.0 100.0 2003 93.2 88.6 99.5 97.4 97.9 96.9 100.0 100.0 2004 100.0 93.7 100.0 99.5 100.0 97.9 100.0 100.0 Column I - signed contracts Column II - completed contracts Source: Own calculations based on PCU quarterly reports. Based on the table above as of the end of 2004 the process of construction of water supply systems was most advanced and nearest completion. Only 30 contracts remained to be completed (4.1% of the total number), of which 27 contracts concerned road investment projects. Training In November 2001 an agreement for conducting training concerning execution of infrastructure investment projects for local government officials was signed with Research Triangle Institute (RTI) company. The company without delay prepared training materials and in December 2001 conducted a pilot training. However, the materials did not gain acceptance of the project team. The necessity to correct the materials resulted in the delayed start of the training. The materials were approved in April 2002, in May a pilot training was conducted, and the training proper began in June 2002. The training ended in March 2003. The report on the progress of training was accepted in the middle of 2003. Evaluation of activities by the beneficiaries Taking the results of the survey concerning opinions of beneficiaries about operation of the Project as the basis for the evaluation, the main objection with regard to the Project, which is evidenced by the highest percentage of respondents' answers, concerns the too long time of waiting for the money. In the interviews often an objection occurred that the incurred costs refund option is a serious problem for many gminas. There were some suggestions for bridging projects (without specifying the form), giving gminas an opportunity to endure the period until Project funds are received. This is reflected by the relatively high percentage of indicated answers “financial assembly” as an important problem with regard to participation in the RDP (we think that this is also a problem in case of other support projects). The percentage of answers indicating the too low level of investment financing was at a similar level. However, looking back in time the local government officials stated that the necessity to provide own contribution may be a serious barrier for some gminas but for most of gminas this is a strong impulse to act and mobilize. The beneficiaries also raised objections with regard to the long period between submitting an application and its acceptance and approval for execution. Such situations may lead for example to expiry of required permits and the necessity to follow again the whole procedure for obtaining them (the survey shows that such situations did occur). Some gminas stated that they had applied for money to be taken from many other sources; some gminas, however, adopted a strategy to wait for those particular the RDP funds. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 321 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The obtained distribution of beneficiaries' answers classifying the main problems of the RDP shows that the assistance in preparing the applications can be evaluated positively. The assistance in this respect was highly appreciated. The contribution of the Project (we mention only Component C here) to the process of improving the institutional environment supporting the development of rural areas can be evaluated very positively with regard to this aspect. Table 105. Biggest problems with obtaining funds Biggest problems with obtaining funds from the RDP Percentage indicated Finding information about such funds 9.3 Preparation of application 21.3 Not clear instructions for preparing an application 10.7 Complicated bidding procedures 13.3 Financial assembly, so as to ensure funds for investment project execution 26.7 Problems with assistance to prepare an application 4.0 Long time to wait for the funds 33.3 Too low level of investment financing 26.7 There was nothing difficult 5.3 Other 2.7 TOTAL 100.0 Source: PBS 2005 survey. The beneficiaries also questioned the sense of the reports, especially the frequency of reports and the requirement to submit reports even if nothing changed. (“The issues connected with reporting might be the only problem, because this is a kind of a novelty, monthly material and financial reports etc., it was necessary to adhere to the time limits. Observing those time limits was a problem”.) The beneficiaries did not raise any serious objections with regard to the “level of bureaucracy” (“the bureaucracy was at a standard level; in other support projects the bureaucracy was more developed. Here we mean the number of enclosures and the volume of applications etc., and in this project this was generally fine” – PBS 2005 survey. The opinions of beneficiaries given below (and many more similar opinions can be found in the gathered survey materials) can be the best recommendation for the Project – “I would be glad to have again something like the RDP from the World Bank. This is really money to take...”; “This was a really well prepared project. It's a pity that it's almost over...” – PBS 2005 survey. 5.3.4 Conclusions and recommendations for the future Conclusions The goals and activities of Component C were adequate to the scale of problems arising from the condition of the rural infrastructure. The very low standard of infrastructure in rural areas (taking rural areas in the EU countries and urban areas in Poland as reference) and the great geographical imbalances fully justified the financial support for infrastructure under the RDP. 322 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The Project was executed in all 16 voivodships of Poland, which we judge as a positive fact. We regard the strategy of common participation (i.e. of all gminas) in Component C as the right one, first of all due to the possibility to select optimum investment projects in the whole Poland for financing. In the situation when funds are limited, this issue is of key importance. The earlier project supporting infrastructure development (World Bank's loan ASAL-300). The number of applications (and also their value) which were received from individual voivodships was correlated with the infrastructure situation; we had an uncomplicated relationship – the lower the infrastructure level, the bigger the demand for investment financing. The biggest number of beneficiaries was located in the following voivodships: świętokrzyskie, and next wielkopolskie, małopolskie, mazowieckie and kujawskopomorskie. The smallest number of beneficiaries was in the following voivodships: śląskie, opolskie, dolnośląskie, lubuskie (apart from the lubuskie voivodship these are voivodships with a relatively most favourable situation with regard to technical infrastructure). Rural gminas prevailed (they accounted for 60%) among the beneficiaries, and they were the most numerous potential group. The quantitative achievements of Component C can be assessed as satisfactory in comparison with expected results, although the obtained effects are different for individual activities, and also the structure of discrepancies is varied. As a result of the completion of investment projects over 13.3 thousand water mains service connections, and 22.3 thousand sewerage service connections have been created, and also 4 waste disposal sites (used by 23.1 thousand beneficiaries) and 236 sewage treatment plants (of which 213 in farms) have been built. The total number of recipients of different types of services created as part of the RDP is approx. 60 thousand households and various entities. As a result of the executed activities 5 940 new jobs have been created, which is 26% of the target (i.e. planned) number of jobs. The increase in the length of water supply systems built under the Project was proportional to general effects accomplished in the whole Poland during the RDP, and the increase in the number of new users was a little higher. Nearly 90% of new connections to water supply systems in the group of gminas participating in the Project resulted from investment projects under the RDP. In comparison with gminas not participating in the Project (average value for the group) the rate of growth of water mains service connections was twice higher than in the group of not participating gminas. The increase in the length of water supply systems and in the number of new users created as part of the Project was higher than effects accomplished in the whole Poland in the years of the RDP duration (considering proportions of participation with regard to the accomplished effect). The percentage of service connections created in the group of gminas participating in the Project was 67% of the total increase for the whole group of gminas. The rate of growth in this respect in comparison with the group of gminas not participating in the RDP was higher by 30%. The influence of activities undertaken under the Project on changes in disproportions between the number of water mains service connections and sewerage service connections was an important element of evaluation of the RDP. The conducted analysis shows that the Project's activities contributed to a decrease in the difference of water mains service connections and sewerage service connections for the whole Poland, and therefore the Project had an influence on improvement of the water supply and sewerage facilities, and so indirectly had influence on improved living conditions of the rural population. Without the Project's contribution the disproportion in this respect would be higher by 10.5% than the actual one (for Poland as a whole). Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 323 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The fact that the disproportion in this respect in the group of gminas covered by the RDP decreased by 30% is particularly worth emphasizing. This means that without the participation of the Project's products the difference between the number of water mains service connections and sewerage service connections would be greater by 40% in comparison with the difference in 2000. We emphasize especially the contribution of the Project to an improvement of conditions in this sphere, because based on conducted analyses the situation in this sphere is very serious, and in some regions we might say that it is alarming. The local governments are especially worried about the quality and effectiveness of local sewage collection systems which to a large extent with regard to utility are a kind of “mock” elements. This is firstly related to the low level of awareness of the rural population, and secondly related to the high costs of maintenance of these local systems. In case of this activity the costs connected with sewage discharged by households decreased. The varied level of subsidies applied in Component C for individual activities can also be regarded as fully justified. Assigning a “bonus” to the “sewerage” activity in the form of a higher subsidy than in case of the “water supply” activity in consideration of the problems above was a correct decision. Maybe these criteria to a certain extent contributed to obtaining a positive effect of the improved ratio between the number of water mains service connections and sewerage service connections. In Component C four activities were executed; the interest in one of them concerning organized (waste) management was very small. Perhaps in this case it would be better not to initiate activity in this sphere and rather to concentrate the funds on more urgent needs and development priorities of gminas, which are reflected by the revealed demand for individual types of activity. In case of the three remaining activities their adequacy with regard to the beneficiaries' needs and their usefulness (which in this case also refers to the waste disposal activity) should be judged as positive. The survey covering local government officials shows that without financial support provided by the RDP it would not be possible to execute nearly two thirds of the infrastructure investment projects in accordance with the plans concerning the scale of the venture and the completion date. Lack of funds from the RDP would mainly result in delays in execution, which in many cases may mean (as indicated by the self-government officials) an increase in investment costs. In gminas which did not receive financial support from the RDP 23% of the planned investment projects have not been executed at all. Nearly 60% of the executed investment projects were delayed due to insufficient funds available, and one in three investment projects has been executed on a smaller scale. The interviews conducted with persons (gminas offices employees) in charge of the infrastructure investment projects execution show that the RDP funds received by a gmina, even in case of investment projects covered by funds from other sources, did play a significant role in infrastructure development in the gmina, because the RDP funds gave an opportunity to carry out several investment projects concurrently, which definitely speeded up the process of infrastructure creation. The Project was carried out in cooperation with other projects (mainly with the program SAPARD, PHARE), and enhanced their positive influence on the rural infrastructure development process. A synergy effect has occurred between the Component C and other projects (other sources of funds for rural infrastructure). 324 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The usefulness of the completed investment projects has been very highly assessed by beneficiaries. The effectiveness of execution of the goals assumed in Component C has been assessed as positive. The Project has contributed to an improvement of situation in the sphere of rural infrastructure and has enhanced the process of introduction of market orientation in projection and construction of infrastructure investment projects. In the light of the results of the surveys (questionnaires and interviews) and overall analysis of the gathered opinions the external effects accomplished as a result of execution of investment projects as part of Component C can be judged very positively. The infrastructure investment projects have contributed to an improvement in the living conditions of the rural population (including health) and conditions for conducting business operations. The executed investment projects more often generated a decrease in financial burden for firms than an increase. Practically in one in five surveyed firms the standard of equipment used by a firm has improved. The investments in water supply systems had the strongest positive influence on improvement of standard of equipment in firms, whereas in case of a modernization or construction of a new road such a relationship has not been recorded. The fact that the infrastructure investment projects contributed to an increase in the scale of production in 16% of surveyed firms may also have an influence on the positive assessment of Component C; 14% of respondents said that the firm's scope of influence increased, in case of 13% of respondents employment increased in connection with a given investment, and thanks to the Project 28% of the surveyed firms managed to survive a hard time. As a result of the executed projects the investment attractiveness of gminas has increased, which is reflected directly by a greater interest in purchase of building lots and recreation lots. A longer time is needed to fully evaluate to what extent the Component C has influenced the growth of non-agriculture employment in rural areas. The survey shows that the investment projects have contributed to a different extent to the growth of economic activity of the rural population, however, it is not possible to specify actual volume of the activity measured by the number of entities and new jobs. The Component C has contributed to the greatest extent to the improvement in living conditions. The infrastructure investment projects made it possible to achieve an improvement in the standard of durable goods in households and an improvement in technical standard of dwellings. They have also contributed to an improvement in health of the rural population and appearance of villages and also social activity of the rural population. The very good cooperation with ARMA Regional Office and VPIUs in the opinion of local government officials, mainly with regard to competence of the staff and their usefulness in practical aspects in application preparation stage and project execution has brought about acquired experience resulting in greater activity of gminas in execution of investment projects using other funds from support funds sources. The Project and its high quality structure (this means the cooperating units) has contributed to strengthening of the process of formation of new “bodies”, i.e. teams of persons who specialize in projects implementation in local governments. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 325 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) The main objections raised with regard to the Project include too long period of waiting for the money and the long time between submitting an application and its acceptance and approval for execution. The beneficiaries also questioned the sense of the reports, especially the frequency of reports and the requirement to submit reports even if nothing changed. A complete novelty of the Project was the training prepared by RTI with the use of on-line Web sites in the area of infrastructure investments in rural gminas. It was a very useful form of continuing the previously conducted training enabling verification and refreshing of the acquired knowledge. The comparison of benefits generated by Component C with costs is highly satisfactory. Recommendations for the future It is worth to improve the existing institutional structures so as speed up the infrastructure project execution (due to its specific character) at each stage. The issue meant here is the time between submitting an application and the decision whether to approve the application. It is also very important to reduce the time needed to wait for the money; if the time is too long, this has a number of adverse consequences for gminas. The conducted analyses show that perhaps it is worth considering once more whether it is better to support to a greater extent than in this Project the activities connected with sewerage construction and those activities in which sewerage and water supply systems are combined. It is very important to support first of all the investment projects which generate the biggest number of jobs; here the criteria and first of all certain guarantees that the number of jobs is not a mere empty annotation placed in an application should be drawn up thoughtfully. Perhaps a support for investment projects having a broader scope of influence (greater number of recipients, lower cost of service) would be a more effective option and perhaps it would be worth to invent new methods to stimulate activity of gminas to get involved jointly in investment activities, where such activities are possible and reasonable. 5.4 Intergmina associations 5.4.1 Survey basis The survey was conducted by PBS in March 2005 using a layer and random sample of 100 persons who were gmina offices officials covered by the RDP. In addition, 24 depth interviews were conducted with gmina and powiat officials involved in execution of the RDP investment projects (C 29 tool). 5.4.2 Results In the questionnaire 63% of persons (i.e. 51 out of 81 persons), who answered the question: Do you know that intergmina associations could apply for subsidies for infrastructure investment projects under the RDP? gave a 'Yes' answer. 326 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) Only six of them said that “while trying to obtain funds from the RDP we, as the gmina, established intergmina associations or the gmina was a member of such an association of gminas”. 45 of remaining persons (i.e. those who as gminas officials knew about the possibility of establishing supragmina associations, but did not participate in them) explained by selecting appropriate answers the reasons for lack of such associations, namely: • no such need - 44%, • specific character of the task (local project) – 27%, • lack of joint goals – 22%, • other reasons – 13%. Note: the total exceeds 100%, because giving more than one answer was allowed. Six officials from gminas which participated in supragmina associations mentioned the following initiators/reasons for such associations: • the Project itself – 2 persons (33%), • other gmina – 1 person (16.7%), • other situation – 3 persons (50%). The results show that there are few initiatives to establish supragmina associations in order to obtain benefits arising from economy of scale or obtain a sufficient critical mass with a view to implement infrastructure investment projects. The lack of such needs or lack of awareness about such needs or perhaps potential benefits arising from establishing supragmina associations seems to be the main reason. In order to better understand this phenomenon a number of questions were asked in indepth interviews conducted during talks with gminas officials in charge of infrastructure investment projects (tool no. 29). Nearly all respondents did not feel a strong need to establish supragmina associations, and they regarded infrastructure investment projects as their local affair. They also did not make attempts to establish such associations due to fears connected with organizational and settlement-related problems during the investment and after its completion. The supposed inborn individualism of Polish people was even mentioned as the reason, and the “swallows say that partnerships are bad” proverb was quoted to confirm the thesis. It was also stated in some cases that the “racing and unhealthy competition” among gminas was the reason. No specific incentives (preferences) were noticed in the Project for organizations to combine so as to establish associations, but some interlocutors said that such preferences existed in other projects, especially EU ones, where it is easier to obtain financing for joint projects. According to several interlocutors this even is a kind of a trend in case of EU projects. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 327 Final Evaluation Study of the Rural Development Project – Component C Beata Pięcek, Ph. D., Jarosław Malec (p. 5.4) As a result, in case of other projects ISPA, LIDER or IOPRD such initiatives were undertaken. In other projects the high investment cost threshold, exceeding the possibilities of a single gmina and a greater chance of acceptance of a joint project, or the minimum material scale of an investment, for example the number of inhabitants using it was the reason for establishing supragmina associations. Creating joint projects might be encouraged by awarding additional points for a supragmina character of a project (such additional points occur for example in IOPRD). Also a higher percentage of funds allocated to an investment than in case of a single gmina would be an important incentive for gminas to combine and establish associations (many interlocutors emphasized this). 5.4.3 Conclusions Joint supragmina projects under the RDP concerning infrastructure investment projects were very rare, and the reasons were as follows: • gminas did not feel the need for such activity, • according to the respondents the incentives (such as additional points for applications, lower required own contribution in financing) for such initiatives were not sufficient, • gminas were afraid of organizational and accounts settlement-related problems, • gminas compete one against another. However, the experience gained from other projects shows that such initiatives are undertaken if investment projects due to their very nature are of supragmina character or in the projects there are explicit provisions enforcing such associations (requirement of appropriate scale) or stimulating such associations (preferences for joint initiatives). 328 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka 6 Evaluation of the impact of external financing from the Rural Development Project resources on the investments realized by local governments Krzysztof Kamiński, Dorota Rudnicka Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 329 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka 6.1 Identification of the subject of evaluation The Component C and Sub-Component B2 basic indicators describing the social and economic situation in the investment realization area, measured before the start and in the course of investment realization activities in order to estimate the changes, but not being an effect of the investments, have been presented in the respective chapters. A measure of the intended objectives, means, obtained products, effects and other parameters (e.g. economic, social, environmental) are the performance indicators. Those indicators were established before or at an early stage of the Project in order to monitor its implementation and to evaluate the results against the earlier objectives. The indicators include the input, product, result and impact indicators. 6.1.1 Input indicators The input indicators, related to the budget allocated to individual investment categories, have been used for monitoring progress in liabilities and payments. Basic data: • financial scope of investment – progress status • investment expenditures planned and borne 6.1.2 Product indicators Product indicators – the activity-related indicators were measured in material units: length of the constructed roads, sewage disposal and water supply systems, number of the water conditioning and sewage treatment stations and waste dumps, also as a percentage of the tasks fulfilled. Basic data: • material scope of investment – progress status 6.1.3 Result indicators Result indicators, corresponding to direct and immediate effects of the investments in Component C, provide information on the changes related directly to beneficiaries. Basic data: • number of the investment users • degree of saturation with infrastructure • directly delivered goods. 6.1.4 Impact indicators The impact connected with the effects of investments in Component C exceeds the immediate effects to direct beneficiaries and includes the effects which will appear after some time but are directly connected with the investment. The impact indicators cover long-term effects for broader population. Basic data: 330 • number of new jobs, • number of new business units. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka 6.2 Evaluation of the impact of external financing from the RDP resources on realization of investments Evaluation of the impact of external financing from the RDP resources on the investments realized by local governments was based on the comparative analysis of the investment realization indicators in units participating in the Project and those not participating in the Project but using other financial means – own or acquired from other projects. The evaluation covered the following aspects: • status and material scope of investment, • planned and borne investment expenditures, • financing sources, • effects measured by the level of infrastructure saturation, • number of the investment users (number of households in the system, number of business units and public buildings served by the system), • unit investment costs, • number of new jobs, • number of new business units. 6.2.1 Evaluation of the projects co-financed from RDP Evaluation of the realization of RDP co-financed projects was based on data contained in the reports of the Project-participating units and in the MIS database, statistical information and detailed questionnaires (Investigation in accordance with Tool 30, 34 - an Appendix to the Report). Financial scope and sources of finance In the Project Appraisal Document the total Component C budget, including all its elements with training and other projects, was planned at USD 194.89 million. Total planned Sub-Component B2 budget, including the teaching process improvement tasks, amounted to USD 24.67 million. The planned share of investment costs in the overall Component budget was the following: • in Component C 91,0% • in Sub-Component B2 40,5% The following table derived from the project documents presents the planned investment budget subdivided into the Bank means and local government means (including own and commercial money) Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 331 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Table 106. Anticipated investment task budget Bank money (millions of USD) Item Local government (millions of USD) Total (millions of USD) Infrastructure investments in rural areas, Component C 54,54 122,85 177,39 Infrastructure investments in rural areas, SubComponent B2: 4,19 5,78 9,97 - school repairs 3,09 4,27 7,36 (resources from the not realized „school transport” task) 1,10 1,51 2,61 Total investment: 58,73 128,63 187,36 - repairs and equipment of day-rooms Source: PAD Operational Manual Figure 59. Planned finance source share in Sub-Component B2 Word Bank means Środki Banku 42% Światowego 42% Środki własne Beneficiary’s own beneficjenta means 58%58% Środki Banku Światowego Środki własne beneficjenta Source: PAD Operational Manual Figure 60. Planned finance source share in Component C Środki Banku Word Bank means Światowego 31% 31% Środki własne Beneficiary’s beneficjenta own means 69% 69% Środki Banku Światowego Środki własne beneficjenta Source: PAD Operational Manual 332 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Table 107. Financial scope of the investment tasks Investment Water conditioning stations Planned costs Actual costs Percentage of actual costs in planned costs Millions of PLN Millions of PLN % 76 47,7 62,8% 301 210 69,8% 7 5,4 77,1% Roads 201 202 100,5% Total C investments: 585* 465,1 79,5 31,5 255,8 Water supply network Sewage treatment plants Sewage network Waste dumps Upgrading the school building condition (7,36 millions of USD) (59 millions of USD) 813,4% 11,2 25,8 (2,61 millions of USD) (5,8 millions of USD) 230,4% Total B2 investments: : 42,7** 281,6 659,48 Total: 627,7 746,7 118,9% Repairs and equipment of day-rooms * at the application stage **from PAD, converted at the rate: 1 USD = 4.2730 PLN Source: Costs in Component C acc. to the RDP Components Monitoring Table, PCU data, as of 31.03.2005 Costs in Sub-Component B2: values in USD acc. to the RDP Components Monitoring Table, as of 31.03.2005; values in PLN acc. to MIS, PCU data as of 31.03.2005 Figure 61. Actual percentage share of finance sources in Sub-Component B2 Środki Banku Word Bank means Światowego 37% 37% Środki własne Beneficiary’s beneficjenta own means 63% 63% Środki Banku Światowego Środki własne beneficjenta Source: PCU data, as of 31.03.2005 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 333 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Figure 62. Actual percentage share of finance sources in Component C Środki Banku Word Bank means Światowego 38% 38% Środki własne Beneficiary’s beneficjenta own means 62% 62% Środki Banku Światowego Środki własne beneficjenta Source: PCU data, as of 31.03.2005 Table 108. Use of the World Bank means for investment tasks Component B2 Task Data School repairs Investments School and day-room equipment Rural infrastructure (non-water supply) C investments Investments Rural infrastructure (water supply) investments Total Credit (millions of EUR) 23,84 Credit (millions of PLN) 98,69 Credit (millions of EUR) 1,50 Credit (millions of PLN) 6,16 Credit (millions of EUR) 41,28 Credit (millions of PLN) 165,97 Credit (millions of EUR) 2,95 Credit (millions of PLN) 11,68 Total: Credit (millions of EUR) 69,57 Total: Credit (millions of PLN) 282,50 Source: PCU data, as of 26.04.2005 334 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Figure 63. World Bank means allocation by the type of infrastructure investment Rural infrastructure investments (w ater supply) 4% School repairs 35% Rural infrastructure investments (nonw ater supply) 59% School and dayroom equipment 2% Source: PCU data, as of 26.04.2005 Figure 64. Co-financing of investments from the World Bank resources in the individual voivodships (provinces) 25 19,3 20 16,9 14,1 15 13,6 13,0 12,6 13,4 11,4 9,7 9,5 8,9 10 6,6 5 17,6 16,4 15,7 6,0 4,8 3,1 9,1 2,8 3,3 2,5 3,0 3,0 3,3 3,5 4,4 8,7 8,2 8,3 5,6 4,1 Wielkopolskie WarmińskoMazurskie Świętokrzy skie Śląskie Pomorskie Podlaskie Subcomponent B2 Zachodniopomorskie Component C Podkarpackie Opolskie Mazowieckie Małopolskie Łódzkie Lubuskie Lubelskie Kujawsko-Pomorskie Dolnośląskie 0 Source: PCU data, as of 26.04.2005 Investments were financed mainly by the local governments. Financing of investments within the RDP was subject to the Public Finance law regulations. Basic costs borne by the beneficiaries include, first of all, costs connected with construction and modernization of the objects. Only in few cases additional costs were incurred by some beneficiaries due to almost half-year long waiting, at the initial stage of the Project, for the Bank money. In several cases the investigated gminas took commercial credits to pay the work executors. Later in the Project the problem was not so serious (waiting time did not exceed a month) and did not generate additional costs. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 335 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Apart from the investment realization costs, the investment beneficiaries are charged with the object operating and maintenance costs. In most of the investigated gminas, however, it was not possible to determine those costs for the RDP financed objects as the objects were usually an element of a system. Within Component C some 79.5% of the money anticipated before biddings were spent. Differences between the planned and actual costs at that stage were caused by the following main reasons: • differences between the cost estimates before bidding and the contract signed with the winning tender (revaluation of the investment), • reduced scope of investment (resignation after application), • additional work which turned out to be necessary during realization of the investment, etc. Within Sub-Component B2 much more money was spent than it had been planned for in PAD before the investment started. In May 2003 it was decided to extend the investments from 7 voivodships to 16 voivodships, with money taken from the reserve fund of the project (free reserve and returns from other Components). As far as the limit of loan funds specified in the Loan Contract is concerned, the total level of funds absorption reached 96.93% in Sub-Component B2 in March 2005, following re-allocation. In the SubComponent B2 investments 99.5% of the planned resources were spent. At the time of Report writing, realization of the material scope has reached 100%, but accounts have not been completed yet. Figure 65. Sub-Component B2 investment costs in voivodships 70 60 16,4 50 14,1 40 8,2 4,4 5,6 World Bank means Wielkopolskie WarmińskoMazurskie Św iętokrzyskie Pomorskie 4,1 24,1 21,6 3,0 3,3 14,8 11,6 8,8 10,1 12,6 Zachodniopomorskie Beneficiary's ow n means 25,9 Podlaskie 3,0 7,8 Opolskie 14,9 Mazow ieckie Lubelskie Kujaw skoPomorskie Łódzkie 9,2 Lubuskie 2,8 3,3 7,4 6,2 0 Dolnośląskie 6,0 Śląskie 39,7 3,1 22,2 9,1 44,7 Małopolskie 20 10 9,5 8,9 Podkarpackie 30 Source: PCU data, as of 26.04.2005 336 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Figure 66. Component C investment costs in voivodships 70 60 13,6 12,6 16,9 40 30 19,3 15,7 50 17,6 13,0 4,8 20 10 23,1 13,4 11,4 48,3 37,1 36,9 6,6 31,9 14,4 46,6 40,7 29,5 2,5 8,3 8,3 37,6 22,9 3,5 8,9 33,2 8,7 23,5 27,0 Ku ja w D ol no śl sk ąs oki Po e m or sk i Lu e be lsk ie Lu bu sk ie Łó dz M ki ał e op ol sk M ie az ow ie ck ie O po Po lsk dk ie ar pa ck ie Po dl as kie Po m or sk ie Śl Ś ą W sk w ar ię ie to m kr iń sk zy osk M ie az ur W s Za k ie lko ie ch od po ni ls op ki e om or sk ie 0 9,7 Beneficiary's own means World Bank means Source: PCU data, as of 26.04.2005 Material scope In the Component C investments, altogether 722 contracts were concluded (source: Monitoring Table, PCU, as of 31.03.2005), including: • construction and modernization of the water conditioning stations and water supply network: 97 contracts, • construction and modernization of the sewage treatment plants and sewage network: 194 contracts, • building of waste dumps: 4 contracts, • construction of the powiat and gmina roads: 427 contracts. In the Sub-Component B2 investments, altogether 3397 contracts were concluded (source: MIS, as of 31.03.2005), including: • school repairs: 1936 contracts, • equipment and repairs of school day-rooms and class rooms: 1461 contracts. The following tables present realization of the RDP co-financed projects. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 337 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Table 109. Scope of the investment tasks Unit Planned products Actual products Percentage (%) of actual products comparing with plan object 14 14 100% km 790 923 116,8% Sewage treatment plants object 351 236 67,2% Sewage disposal network km 916 1037 113,2% object 5 4 80,0% km 557 734 131,8% - number of repaired schools object 100 1885 1885,0% - repaired class rooms object 348 4126 1185,6% 259 498,1% Investment Water conditioning stations Water supply network Waste dumps Roads Upgrading of school objects Repairs and equipment of day-rooms - number of repaired day-rooms object 52 - number of equipped day-rooms (and zero-level class rooms) object 111 682 (491) 1056,8% Source: PCU data, as of 31.03.2005 Water supply system The „Water conditioning stations” investments include construction and modernization of water conditioning stations and water intakes. Three new plants have been built: 2 water conditioning stations (total cost PLN 1,314,500.) and 1 water intake with water conditioning station (cost PLN 1,261,600). Other objects were modernized or repaired. The water supply network, constructed or modernized, will serve households (with 12,900 connections) and institutions (422 connections). Unit building costs of the water supply network are on the low side of average costs in the country. The sewage treatment and disposal system In the RDP, 213 new farmstead sewage treatment plants have been built in one gmina with 5500 inhabitants. At the application submitting stage 330 such plants had been planned but some potential beneficiaries withdrew their applications. Unit cost of a farmstead sewage treatment plant is PLN 1,767 per (m3/day). Other 23 sewage treatment plants include existing modernized or extended objects and 15 new objects in the area with some 120,000 inhabitants. Unit cost of the new objects is in the range of PLN 6,000 to 12,000 per (m3/d) of treated sewage, depending on the technology used and plant capacity. Unit costs are on the high side of average costs of such objects built in Poland. Line objects are new and modernized objects. The newly built sewage disposal network includes sewage pumping stations as well as 20,850 connections to households and 1,400 connections to other buildings. Unit building costs of the sewage disposal network are on the low side of average costs in the country. Waste dumps Waste dumps are newly built objects. Their total capacity is 138,926 m3. Those waste dumps will be used by 18,600 households and 4,500 other users – public service 338 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka institutions. Unit cost of the investment is in the range of PLN 11 to 93 per m3 of waste dump capacity. Roads Poviat and gmina roads were built within the Component C. In the total length of constructed roads (734 km), applications for 332.94 km were submitted by powiats. In other cases applications came from gmina offices. In most investigated gminas (79%) the investments were qualified by beneficiaries as modernization of existing roads. Average unit cost of a modernized road in those gminas was PLN 236,900 per km. Construction of new roads was declared by 21% of the gminas. Average unit cost of a new road was PLN 485,400 per km. School objects Within the Sub-Component B2 repairs were performed in 1885 schools, including 4126 classrooms and 259 day-rooms. Equipped were 682 day-rooms and 491 classrooms. The repair and investment tasks of the Sub-Component B2 were initially carried out in 7 voivodships (1st phase). In 2003 the initial scope of tasks was extended and all the voivodships were included into the Sub-Component (2nd phase). The estimated number of schoolchildren attending schools included in the Project, who directly benefitted from it, exceeds 300,000. Investments covered by Component C were completed by the end of 2004, with the exception of one project in the road sector. In the first quarter of 2005 financial accounts were made but have not been finished yet. The planned material scope of the investments was kept for the water conditioning stations. The scope of construction and modernization of the water supply and sewage disposal networks was extended. The number of waste dumps was reduced from 5 to 4. All the planned gmina sewage treatment plants have been built, but 117 potential beneficiaries withdrew their applications for building small farmstead sewage treatment plants. It may be concluded that material assumptions of Component C have been fulfilled. In Sub-Component B2, in May 2003 the NSC decided to extend the material scope of investments comparing with the initial plans presented in PAD. Investments were realized in all voivodships. The scope extension was possible by appropriating means for it from the Project reserve fund. The latest contracts were concluded in the first quarter of 2005. At the time of preparing of this Report, the material plans have been fulfilled in 100% but financial accounts have not been finished. In Sub-Component B2, overall material effects have been considerably greater than the initial assumptions, which indicates great demand on the part of beneficiaries for that type of investments. The unit investment costs given in the table below have been determined for this Report as an average value of work for a unit of the respective material effect. The individual investment groups include different work categories: construction, repair and modernization of objects, therefore the cost values in the table are not comparable with unit cost of new buildings. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 339 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Table 110. Cost of investments in the Project per unit material effect Investment Unit cost of investment Unit Water conditioning stations [thous. of PLN/object] 647.9 [thous. of PLN/km] 43.3 Water supply network 3 Municipal sewage treatment plants [thous. of PLN/ m /d] 3.1 Municipal sewage treatment plants [thous. of PLN/object] 1 844.8 3 Farmstead sewage treatment plants [thous. of PLN/ m /d] 1.8 Farmstead sewage treatment plants [thous. of PLN/object] 8.8 Sewage disposal network [thous. of PLN/km] 159.7 3 Waste dumps [thous. of PLN /m of capacity] 0.04 Waste dumps [thous. of PLN/object] 1 353.4 Roads [thous. of PLN/km] 276.7 Upgrading the school objects [thous. of PLN/object] 135.7 Repairs and equipment of day-rooms [thous. of PLN/object] 18.0 Source: MIS unit investment costs, as of 31.03.2005. As the average population density is comparable in all the investigated gminas, whether participating or not in the Project, an average number of new users benefitting from an investment project in a given sector, determined from the questionnaire interviews, was chosen as a measure of the dynamics of investments undertaken and carried out in the years 2001-2004. Table 111. Average number of investment users in one project in a given sector Sector Estimated number of Estimated number of business households benefitting from 1 units and public service objects project benefitting from 1 project Water supply 191.2 8.4 Sewage treatment 107.5 7.3 Waste dumps 4652.5 1116.0 Roads 417.4 31.6 Estimated number of pupils benefitting from repaired classrooms Upgrading the school objects 227.1 Source: PBS survey in March 2005 Information was gathered from the questionnaire interviews in gminas. Number of new jobs It is estimated that new jobs created as a result of the repair and modernization work carried out in Sub-Component B2 have been located mainly in the repair and construction companies. This indicator was not monitored during the Project. 340 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka The impact of Component C investments measured as a number of new jobs available to the direct beneficiaries cannot be assessed immediately. The actual impact of the Component C investments on the social and economic situation will be seen after some time. The number of new jobs, as both direct and indirect effect of the investments, as of 31.03.2005, is given in the table below. Table 112. Expected and actual effects of Component C in terms of new jobs created Expected results Actual results as of 31/03/2005 Poviat and gmina roads 15 100 3 704 Water supply 2 650 821 240 177 Sewage management 5 400 1238 Total: 23 390 5 940 Task Waste management Source: the RDP Components Monitoring Table, PCU, as of 31.03.2005 Final estimation of the „Number of new jobs” indicator is planned for December 2006. 6.2.2 Evaluation of the realization of projects where applications for a RDP financial support had been submitted but projects were not qualified for financing From available data, e.g. results of investigation of a representative group of local government units (investigation in accordance with Tool 33, attached as Appendix to this Report), an investigation was carried out of the realization of projects where applications for RDP co-financing had been submitted but projects were not qualified. In Component C a total number of 1013 projects (the MIS database registers costs of 2 projects whose overall value after procurement procedures reached about 1 million PLN). In Sub-Component B2 a total of 1034 projects were not qualified (the MIS database registers costs of 2 projects whose overall value after procurement procedures reached about 123.8 thous. PLN). Most of the submitted to the RDP and not qualified projects were simply not fully prepared for realization. The most frequent reasons of negative verification were lack of valid building permit and/or other documents, shortage of the required own means and applications submitted after bidding not in accordance with the World Bank procedure. Some projects, positively verified, were eventually not included in the Project because in the meantime they obtained financial support from other projects, e.g. PHARE INRED (most projects were in the water supply - sewage management sector), PHARE ODBUDOWA (reconstruction of the infrastructure after the 2001 flood, including road repairs), SAPARD (investment support), Restructuring Funds, Environment Protection Funds. 35% of the investigated gminas applied for the investment financial support from other sources. In the investigated gminas, investments were realized in all the RDP covered sectors. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 341 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka The questioned respondents declared that realization had been undertaken of more than 77% of projects submitted to RDP, of which 31% were realized on a smaller scale. Rejection from RDP caused delays of investments in almost 57% of the investigated gminas. In most cases the investments were realized in 2004. As the average population density is comparable in all the investigated gminas, an average number of new users benefitting from an investment project was chosen as a measure of the effects of investments undertaken and carried out in the years 2001-2004. Table 113. Average number of investment users in one project in a given sector Sector Estimated number of Estimated number of business households benefitting from 1 units and public service objects project benefitting from 1 project Water supply 233 33 Sewage treatment 388 43 Waste dumps n.a. n.a. Roads 563 68 Estimated number of pupils benefitting from repaired classrooms Upgrading the school objects 141.8 Source: PBS survey in March 2005 Information was gathered from the questionnaire interviews in gminas. 6.2.3 Evaluation of the realization of projects in the local government units which did not apply for a RDP financial support From available data, e.g. results of investigation of a representative group of local government units (investigation in accordance with Tool 35, attached as Appendix to this Report), an investigation was carried out of the realization of projects in units which had not applied for RDP co-financing. The largest group of the gminas were those with 6,000-10,000 inhabitants, 30% were gminas with 1,000-5,000 inhabitants. 60% of the population live in the rural areas. In the investigated gminas, investments were realized in all the RDP covered sectors. The main reason of not applying for the RDP co-financing were applications submitted to other support funds. Almost 90% of respondents benefitted from the European Union accession funds: ISPA, PHARE and SAPARD. 38% of the gminas used means provided by the international financial institutions. Less popular were the Structural Funds and the Rapid, Struder and Inred programs. Almost 60% of respondents are of the opinion that without the external co-financing it wouldn’t be possible to realize the investments or they would be delayed or would have to be realized in stages. 342 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka An average number of new users benefitting from an investment project was chosen as a measure of the effects of investments undertaken and carried out in the years 2001-2004. Effects of the projects not co-financed from RDP are presented in the table below. Table 114. Average number of investment users in one project in a given sector, not co-financed from RDP Sector Estimated number of Estimated number of business households benefitting from 1 units and public service objects project benefitting from 1 project Water supply 644 33 Sewage treatment 260 20 Waste dumps 1978 268. Roads 169 15 Estimated number of pupils benefitting from repaired classrooms Upgrading the school objects 139.3 Source: PBS survey in March 2005 Information was gathered from the questionnaire interviews in gminas. 6.2.4 Indicators The impact of RDP financing on the realization of local government investments was considered taking into account three groups of investments: • RDP co-financed investments • Investments where applications for RDP co-financing had been submitted but were rejected • Investments where applications for RDP co-financing had not been submitted. The source of information presented in the table are first of all the GUS statistics for some selected gminas, where also the questionnaire interviews were performed. The table below contains average values of indicators in the individual sectors calculated for the investigated areas. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 343 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Table 115. Selected indicators for investments realized by local governments in the investigated gminas Indicator Unit Investments cofinanced from RDP Investments not co-financed from RDP RDP co-financing not applied for Applied for, not qualified to RDP co-financing Water conditioning stations (WCS) and water supply network WCS output per person using the water supply network m3/P d 0,00562 0,00131 0,00272 Conditioned water per person per year 3 dam /P year 0,0254 0,0264 0,0247 Length of operating water supply network per person km/P 0,0096 0,0111 0,0094 Length of operating water supply network per unit of the gmina area km/km2 0,786 0,914 0,667 Number of persons per one water supply connection P/connection 4,94 4,44 4,39 Percentage of population using water supply network in 2003 % 78,5 81,1 71,6 Investment expenditures per person using water supply network – construction and modernization of water conditioning stations thous. of PLN/P 0,005 0,004 0,009 Unit cost of water supply network (intake and distribution) PLN/km 94,21 67,49 80,12 3 1,80 1,88 1,84 3 2,98 2,21 1,98 191,2 644 233 3 Net price of 1 m of water - households 3 PLN/m Net price of 1 m of water – industry PLN/m Average number of connected households as an effect of one project households/project Average number of connected business units as an effect of one project business units/project 8,4 33 33 Treated sewage per person per year dam3/P year 0,0175 0,0130 0,0126 Length of operating sewage disposal network per person km/P 0,0024 0,0018 0,0030 Length of operating sewage disposal network per unit of the gmina area km/km2 0,1985 0,1494 0,2117 Sewage collected dam3 /P year 0,0191 0,0142 0,0138 Sewage treatment plants and sewage disposal network 344 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Number of persons per one sewage disposal connection P/connection 8 9 6 Percentage of population served by sewage treatment plants % 44,28 30,97 31,04 Percentage of population using sewage disposal network % 38,17 28,85 29,18 Unit cost of sewage disposal network thous. of PLN/km 257 359 264 Investment expenditures per person using sewage disposal network – construction and modernization of sewage treatment plants thous. of PLN/ person 0,049 0,089 0,060 Net price of 1 m3 of sewage - households PLN/m3 2,63 2,64 2,61 3 2,98 2,80 2,72 107,5 260 388 3 Net price of 1 m of sewage – industry PLN/m Average number of connected households as an effect of one project households/project Average number of connected business units as an effect of one project business units/project 7,3 20 43 Waste per person per year (average) kg 290 290 290 Investment expenditures per person – waste management PLN/P 4,40 3,98 3,80 Gmina budget expenditures per person – waste management PLN/P 0,72 0,60 1,07 Average number of households benefitting from one project households/project 4652,5 1978 b.d. Average number of business units benefitting from one project business units/project 1116 268 b.d. 41 242 26 328 53 240 40 42 57 417,4 169 563 31,6 15 68 Waste Roads Gmina budget expenditures per year for gmina public roads thous. of PLN/year Gmina budget expenditures per person per year for gmina public roads PLN/P year Average number of households benefitting from one project households/project Average number of business units benefitting from one project business units/project School objects Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 345 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Average school building maintenance costs PLN/year 291,0 298,3 305,9 Number of 6-years old children attending zero-classes (in elementary schools) person 14,5 13,0 14,2 Number of pupils using school day-room (gmina government person administered schools) 16,7 16,2 16,3 Number of pupils at school (gmina government administered person schools) 159,8 146,6 148,0 Number of repaired class rooms room 16,3 8,6 8,7 Number of repaired day-rooms room 13,6 12,4 11 Information on gminas Population density P/km2 82 83 71 Gmina budget income per person PLN/P 826 650 1 039 Gmina budget income – state budget subsidies - per person PLN/P 78 52 86 Gmina budget income – acquired from other sources - per person PLN/P 33 26 33 Gmina budget income – own income - per person PLN/P 296 232 402 Source: GUS, 2004 and questionnaire interviews 346 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka The water supply and sewage disposal projects are best documented. The available indicators for roads, waste and schools allow only a rough estimate. Water conditioning stations and water supply network Taking into account that in all three groups of gminas – those co-financed by RDP and not participating in RDP – the population density, average infrastructure development level and average number of persons using one connection are at a similar level, it may be assumed that all those gminas have similar level and structure of the water supply services. In the realized investment tasks, the highest increase in the water supply service development was achieved in the group of gminas which had not applied for the Project means. A comparable although somewhat lower level is in the Project participant group and the lowest level is in the group not qualified to the Project. Comparing the unit costs of the water supply network construction and the investment expenditures in the three groups of gminas, one can see that the Project participant group invested in more expensive objects but more modern and cheaper in operation. Price of 1 m3 of water for households in that group is lowest. However, the average price of 1 m3 of water for industry is the highest, which is not a good prospect for one of the main RDP objectives, i.e. development of business initiatives. It has to be added, though, that those tariffs not always reflect the actual water supply costs for the industry as gminas often apply their own tariff policy not in agreement with economic calculation. Almost 65% of respondents are of the opinion that all the investment objectives have been achieved and more than 70% see the investment as fully compatible with the gmina needs. Sewage treatment plants and sewage disposal networks Having in mind the same conditions as those for the water supply system, it must be observed that in the Project participating group of gminas the sewage management infrastructure is much better developed than in the other groups. Nevertheless, the level of sewage disposal services (44%) is still too low for the expectations, particularly in view of the statutory requirement that each agglomeration with an equivalent number of inhabitants above 2000 should have a sewage disposal network leading to a sewage treatment plant. Costs of the sewage disposal network constructed within the Project was the lowest, which confirms a competitive attitude. The operating costs do not show any significant differences. Availability of the sewage disposal network is not high, but a useful mechanism has been started of investing in the environmentally friendly infrastructure, which so far has not been much appreciated in the rural areas. 80% of respondents in gminas participating in the Project were of the opinion that investment was in full agreement with the actual needs. The most important positive effect was natural environment protection (19%), improvement of sanitary conditions (13%) and improved quality of life (12%). Waste Three of the waste dumps built within the Project are located on the Mazowieckie and Wielkopolskie voivodship territory, which occupy the second and third place in Poland in terms of waste generating. The average number of households using the waste dumps built in the Project participant gminas is twice as high as in those not applying for the RDP Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 347 Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka co-financing. Similarly, the number of business units is four-fold. Only in one gmina not participating in the Project a new waste dump has been built. Very low investment expenditures for that purpose in the gmina budgets confirm that most rural households are not covered by an organized waste collection system and they utilize it on their own, often polluting the natural environment. Development of a network of waste dumps and the waste utilization systems is therefore an important problem not only for cities but also for the rural areas. Roads The highest budgetary expenditures per person for public gmina roads were borne by the RDP non-qualified group of gminas. In other groups those expenditures were somewhat lower, which suggests different levels of construction and modernization work in the overall investment volume in the individual gminas. In the Project participant group of gminas, most investments consisted in modernization of the existing gmina roads (more than 77% of projects). Average road density has not been significantly increased in that territory but the road quality has improved. Owing to the better road quality, the average traffic density is now 30% higher. Almost 90% of the respondents were of the opinion that the new road was fully compatible with the gmina needs. The most important positive effects were the following: improved road safety (100% of respondents), better conditions for business activity (93.2%), improved transport of school children (90.9%), better access to the village (88.6%). School objects Taking into account that the average number of school attending children and pupils is similar in all the investigated gminas, it is worth noticing that in the Project participating group of gminas the number of repaired school compartments is twice as high as in the remaining groups. It confirms the great demand for investments in the improvement of school facilities. 6.3 Summary and conclusions The following may be stated as a summary of the realization of investments by local governments: 348 • The planned investment realization deadlines have been kept. • In the Component C, the actual costs were much lower than costs estimated at the application stage. This is mainly due to differences between the estimated value before bidding and a value contracted with the winning bidder (investment revaluation) and reducing the overall scope of investment due to resignation by beneficiaries from some initially planned tasks. The contract value might have been increased by additional work which appeared necessary in the course of investment realization. Reduction of the material scope of the Project as a whole may be considered small. • In the Sub-Component B2, much more money was spent than had been planned in PAD before the investment started, because in 2003 all the voivodships were included in the Project (7 voivodships originally planned) thanks to transferring the Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of the impact of RDP resources on the investments Krzysztof Kamiński, Dorota Rudnicka Project reserve money to that Component. Therefore, the material effect of the investments was higher than originally planned, in proportion to the financial means actually assigned. • In the starting period of the Project, the main disturbing element of its realization was the prolonged process of transferring money means to the gmina bank accounts. It might take up to half a year from the time an application for transfer of a money installment was accepted to the time the gmina had that money at its disposal. That might cause conflicts with those waiting for payments for work done. In some cases gminas took commercial credits to discharge their liabilities, which was an additional burden on the gmina budget. At a later stage that inconvenience was eliminated and the money transfer time was much shorter. Financing the investments in installments in the course of realization was convenient for the beneficiaries as gminas did not have to engage financial means in the full amount of the expected subsidy. Such a large required own contribution would have been a major impediment and sometimes an insuperable barrier to the investments. • Costs of the objects constructed within the Project are similar to those typical of such objects in Poland. • Unit construction cost indicators of the objects are similar to the indicators in other support projects. • In view of the dispersed and local character of the individual tasks and their relatively small size, the reduction of unit expenditures (scale effect) is not felt. Analysis of data during the final evaluation of the infrastructural investments realized within the Rural Development Project allows to formulate the following conclusions pertaining to the basic Project implementation terms and conditions: • The Project tasks are in great majority of key importance to the functioning of gminas. • The gminas have a real demand for financial support of investments, most of which would not have been realized without the external financing. • Investments covered by the Project are complementary to other programs aimed at improvement of living conditions and business activities in the rural areas. All those programs support each other to achieve a superior objective which is development of the rural areas. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 349 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec 7 Analysis of the social and economic effects Jarosław Malec Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 351 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec The Project had an exceptionally complex multi-subject character rendering many effects difficult to rate in terms of money. As stated in PAD, full social and economic Cost-Benefit analysis based on hard numbers and their projections into the future is not possible – many arbitrary assumptions would have to be made and expensive investigations performed to collect data not obtainable from the available sources. Therefore, like in PAD, the standard cost-benefit analysis (CBA) was left aside and the economic return rate (ERR) was not calculated. It can only be said that for the Component C, where the CBA was performed for each investment project, the return rate was not lower than 10%, which was an acceptance condition for the individual projects. Instead, the following four independent analyses were performed which allow to evaluate the cost effectiveness, productivity and the social and economic impact of the Project. • Analysis of the financial benefits for the state budget from the new jobs created – in accordance with the PAD methodology corrected for the legal conditions and the macroeconomic data for the year 2003. • Qualitative analysis of the costs and the social and economic benefits, from direct investigations. • Comparative (benchmarking) analysis of the project effectiveness, comparing the project costs and effects with similar projects and the market data. • Statistical analysis – investigating the interdependence (correlation) between the Project intensity indicators in Components A, B1 and C (outputs) in the individual gminas and powiats and the key Project result indicators (outcomes). 7.1 Analysis of the financial benefits for the state budget from the new jobs created The financial effects for the state budget from the new jobs created have been evaluated in PAD. The effects consist of two elements: • effects of the reduced unemployment benefit costs; • effects of the increased budget revenue from the income tax and VAT. Additionally, as the wages of the newly employed people were unknown, calculation of the revenue from taxes was made for two wage levels – the statutory minimum wage and an average wage in the country. For the 1997 data, the effects were calculated in PAD at the following level: 352 • effects of the reduced unemployment benefit costs - PLN 2 705 per year per one unemployed person newly employed; • effects of the increased budget revenue from the income tax and VAT – PLN 1 066 per year per one unemployed person newly employed with the minimum wage and PLN 3 314 per year per one unemployed person newly employed with an average wage. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Taking that into account, the state budget benefits from one new job created were in the range (for the 1997 data) from PLN 3 772 to 6 020 per year, depending on the wage level. Assuming the PAD methodology, the state budget benefits from one new job created were determined for the 2003 data (the necessary data were not available for the later periods). The following assumptions were made to evaluate the effects of reduced unemployment benefit costs: • the sums of unemployment benefit, insurance paid from the state budget and administrative costs (for the whole country) were taken from the 2003 state budget performance information published in the Ministry of Finance internet pages (www.mofnet.gov.pl); • the number of unemployed was taken from the GUS data and that number for 2003 was determined as a mean value of the number of unemployed at the end of 2003 and 2002: (3,175,674 + 3,216,958)/2 = 3,196,316. With those assumptions, the unemployment benefit costs (borne by the state budget) of one unemployed person in 2003 amounted to PLN 1 319.55 per year. Details of the calculation are given in the table. Table 116. Unemployment benefit costs of one unemployed person borne by the state budget (based on the 2003 data) Thous. of PLN Unemployment benefits (Poland total) 2 833 587 Insurance of the unemployed (Poland total – government contribution) 907 778 Administrative costs (Poland total) 476 337 Total fiscal expenditures (Poland total) 4 217 702 Total number of the registered unemployed 3 196 316 Annual fiscal cost per 1 unemployed (in PLN) 1 319.55 Source: own analyses. It is worth noticing that the sum of PLN 1 319.55 is a mean value for one unemployed regardless of whether that person still has the right to receive the unemployment benefit. The annual cost in 2003 per one unemployed with the right to receive the benefit was PLN 8,300 (average number of the unemployed with the right to receive the benefit in 2003 was 508,400). The following assumptions were made to evaluate the effects of increased budget revenues from the income tax and VAT: • minimum wage of PLN 800 per month was assumed, based on the regulations in force in 2003 (art. 4 par. 1 of the Minimum Wage law of 10 October 2002 - Dz. U. (Off. Journ.) of 2002, no. 200, item 1679); as in 1997 the respective regulations were changed and „gross” wages were introduced, the basis for calculation was minimum wage minus the obligatory social security premium, i.e. PLN 650.30 per month; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 353 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec • average wage of PLN 2201.47 per month was based on the GUS data published in the internet pages www.stat.gov.pl; as with the minimum wage, this is a „gross” wage, therefore the basis for calculation was average wage minus the obligatory social security premium, i.e. PLN 1 789.60 per month; • the tax-free income was assumed PLN 2 789.89 zł, - based on the regulations in force in 2003, i.e. Personal Income Tax law of 26 July 1991 (uniform text in Dz. U. of 2000, no. 14 item 176 with later amendments); • the income tax was calculated in accordance with the personal income tax law, with the lowest 19% rate; • the increase in revenue is the difference between the wage (minimum or average, depending on the calculation variant) and the tax paid and the average sum per one unemployed person paid from the state budget; • propensity to spend was assumed in accordance with PAD, i.e. 99% in the case of those receiving the minimum wage and 94% in the case of average wage; • increase of expenditure is the product of increase in income and propensity to spend; • food expenditures for those receiving average wage were assumed at 23.8% of income – based on the information published in the GUS internet pages (www.stat.gov.pl) titled „Situation of households in 2003 in the light of the results of household budget investigations” (signal information); as no reliable data were available on the food expenditures in the households of minimum wage receivers, it was assumed for the analysis that food expenditures would be in the same proportion to the expenditure increase as that given in PAD for 1997; • VAT on food was determined in the same way as in PAD for 1997, i.e. as a product of the 7% rate and food expenditure; • non-food expenditures were determined as difference between the total increase of expenditures and food expenditures; • VAT on non-food articles was determined in the same way as in PAD for 1997, i.e. it was assumed that for minimum wage receivers the average VAT rate for non-food articles would be 10%, and for the average wage receivers 15%; VAT value is therefore equal to the product of VAT rate and expenditures for non-food articles; • additionally, value of social security (ZUS) premiums paid by the unemployed who took up new jobs was qualified as a budget benefit as the ZUS budget is supplemented by the state budget means and each ZUS premium paid by a newly employed person reduces the state budget expenditure. With those assumptions, the value of increased fiscal revenue from one unemployed getting new job in 2003 amounted to PLN 3 263 to 10 533 per year, depending on whether the unemployed had received the minimum or average wage. Details of the calculation are given in the table. 354 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Table 117. Annual fiscal benefit of increased taxes and social security premiums paid by the unemployed persons who found jobs – minimum and average annual wage (based on 2003 data) Minimum wage (PLN) Average wage (PLN) Gross minimum wage 9 600 26 418 Minimum wage minus ZUS premium 7 804 21 475 ZUS premiums paid by the unemployed who found jobs 1 796 4 943 Tax free income 2 790 2 790 Taxable income 5 014 18 685 953 3 550 Income increase (wage minus tax minus average unemployment benefit for each unemployed) 5 965 17 038 Propensity to spend 0.99 0.94 Expenditure increase 5 905 16 016 Food expenditure 2 555 4 533 179 317 3 350 11 483 VAT on non-food articles 335 1 723 VAT benefit 514 2 040 3 263 10 533 Income tax VAT on food Non-food expenditure Tax benefit + increased ZUS premium benefit Source: own analyses. The fiscal benefit of creating one new job is the sum of spared expenditures for the unemployment benefit and increased tax income and social security premiums. Depending on the wages received by the newly employed person, the value of annual fiscal benefit of creating one new job, for the 2003 data, was in the range of PLN 4 582 to 11 852. Details are given in the table below. Table 118. Annual fiscal benefit of creating one new job – minimum and average annual wage (based on 2003 data) Minimum wage (PLN) Average wage (PLN) Increased taxes and ZUS premiums 3 263 10 533 Spared cost of the unemployment benefit 1 320 1 320 Total fiscal benefit 4 582 11 852 Source: own analyes. Assuming that creating new jobs reduces the number of unemployed accordingly, we estimate that the fiscal benefit due to creating 10 265 new jobs as a result of the realization of RDP amounted to PLN 47.03 million to 121.66 million depending on the wage received by a newly employed person. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 355 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec 7.2 Direct investigation analysis of the costs and the social and economic benefits 7.2.1 Investigation method Data for the presented analysis were acquired in the following ways: • By selection and synthesis of information contained in 250 interviews with the project beneficiaries and with persons involved in its implementation. Among those interviewed were persons working at all levels of the RDP organization, beginning from the gmina offices up to the members of the National Steering Committee and ministry employees. Information was also gathered in that form from the creditor representatives (World Bank) and various institutions involved in the execution of RDP (e.g. training establishments). This information was primarily of a qualitative character. • By the analysis of results of more than 5000 questionnaires collected by means of a number of research tools. The questionnaires were filled in by the project beneficiaries, institutions involved in the RDP and the gmina officials. This information was primarily of a quantitative character. The method used allows to perform mainly a qualitative analysis of the social and economic impact of the Project. All the quoted numbers should be treated with caution as they are only a result of questionnaire investigations and interviews. The interviewed persons presented their own subjective opinions on the numerical data and indicators, most often without any documents or preparation for talking about the numbers. With this method of collecting data, some information was simply not attainable. In most cases that applied to detailed data which the respondents were unable to recollect during the interviews. The level of competence and credibility of the interviewed persons was also differentiated and difficult to assess. More reliable may be estimates of costs connected with the Project participation, less reliable are the opinions on the economic matters of a gmina or powiat. Many interviewed persons, aware of their poor knowledge, refrained from giving any numbers. Nevertheless, those opinions are a valuable source of information on the social and economic impact of the Project, in qualitative terms. 7.2.2 General evaluation of the social and economic effects of the Project Evaluation of the RDP social and economic effects should be considered in two aspects. The first is a short-term perspective. In the opinion of all the investigated groups (representatives of gmina authorities, enterprises, households) the living conditions improved considerably in a short period of time, particularly in the areas where the infrastructural investments were realized, and also a general situation of the RDP participating gminas was improved. The effect of new jobs created, relatively easy to distinguish, appeared in those gminas where the Component A – micro-loans was implemented. According to the gmina authorities, the main benefit of the realization of RDP is the increased investment attractiveness (see Figure 1). New sources of income for the gminas were a relatively rare outcome of RDP – only in every fifth gmina. 356 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Figure 67. Benefits for gminas resulting from RDP other benefits 6% new sources of income 19% increased personal income 16% improved conditions of teaching 4% increased investment attractiveness 29% development of tourism 26% Source: Own research A common conviction of the beneficial economic effects of RDP should be reflected in the long-term outcomes. At present, the time scale is too short to evaluate all the quantitative effects, e.g. better chances of finding jobs by students attending better equipped schools or an impact of a well operating sewage disposal system on the tourist attractiveness of a location. The project should not be evaluated only through the quantitative indicators. The quantitative elements were treated as a starting point for transformation of the rural areas and, in the opinion of our respondents, such an effect is expected. Therefore, a great deal of attention was paid in the analysis to establishing the economic correlations generated by the RDP. A very important effect of the project was learning how to absorb the support funds. Such knowledge was acquired on many organizational levels. For instance, in many cases private training companies (performing orders within RDP) learned how to use the European Social Fund means132. Gminas learned that the support funds might be used effectively. The RDP procedures were for the local governments a valuable lesson how to apply for various support means, including structural funds. More than 80% of the local governments indicated that effect. If we take into account the local development plans, necessary for getting support financing and prepared within Sub-Component B1, it turns out that RDP has initiated a multiplier effect which will result in faster economic development through more efficient absorption of the support funds. Also important in this context is the psychological effect: the RDP participating gminas, encouraged by the success, took to preparing more technical and construction projects. In the opinion of local government officials, as an effect of the realization of RDP the offices learned how to use the budget more effectively, how to improve the organizational structure and how to reduce the office functioning costs. In the whole country, RDP has left an institutional legacy for the EU programs, e.g. in the form of the Regional Steering Committee experience. All those factors will contribute in the future to faster economic development of rural areas. RDP has also had a meaningful impact on the labor market. Very often underlined is the effect of Sub-Component B1 consisting in rousing many unemployed from indolence and 132 In many cases contracts realized within RDP allowed private companies to survive on the market. The economic decline brought tender prices down to a very low level and these contracts were an incentive for further functioning. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 357 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec apathy caused by many years of inactivity and of teaching them how to look actively for jobs133. The requalifying program caused a multiplier effect. The trained unemployed made a good example for others and popularized good patterns of behavior in local environment (e.g. greater mobility). Also the Voivodship Labor Offices shifted the center of gravity of their activities, thanks to RDP, from administrative functions to active employment policy (e.g. by contacts with employers and recognizing their needs). The organizational potential of labor market institutions increased significantly. Qualifications learned by the unemployed will in longer term cause reduction of social costs (e.g. the unemployment benefit payments) and will help the present unemployed to contribute to the local economic growth. It has to be noticed, however, that the assumptions about new jobs were too ambitious. Comparing with the year 2000, among the RDP gminas the unemployment rate decreased only in every seventh gmina. Such situation is most often connected with the general unfavorable economic situation in the country and the resulting liquidation or restructuring of businesses and job reduction. The RDP means caused increased number of jobs in every third gmina, where on average 13 new jobs were created and an estimated unit cost of a work place was PLN 15 770. 7.2.3 Economic effects of Component A The most important effect of micro-loans was starting of business activities by the borrowers. Every second loan was used for starting new business. Also a positive impact was observed on functioning of the existing business units – turnover, profitability and employment. For majority of the borrowers (73%) obtaining the loan did not cause any additional costs. Most frequently mentioned additional costs were traveling costs (73%) and additional work time (39%). It is worth noticing that those costs increased proportionally to the distance between the borrower and the loan advisor locations. Average traveling cost per micro-loan was PLN 153. The delays in loan payments caused lost profit costs but that effect was not frequent. Two thirds of the borrowers were of the opinion that the micro-loan changed their financial, social or professional situation for better. The borrowers were not the Component A knowledge multipliers as the loan granting period was relatively short. Microloans had a direct impact on creating new jobs. As a rule, they were contracted by persons starting their business activity. By contracting a PLN ten or twenty thousand loan, 2 – 5 new jobs were created, mainly for the borrower’s family members. 7.2.4 Economic effects of Sub-Component B1 Training As an effect of the training, 46% of those questioned found jobs in their place of residence within 3 months of the end of training. If those are included who found jobs in a different location then the indicator increases to 48.8%. Only 2% of the questioned started their own business activity after training. Most of those questioned (63%) considered the type of their training compatible with the new job, which significantly helped to find it. Unfortunately, the character of started activity or new job was in most cases (71%) temporary. Participation in training allowed some of the beneficiaries to go abroad. A positive effect of such case is the observed transfer of foreign earnings to the country. Standard of living has improved, but only of the active training participants. 133 Attractiveness of the requalifying programs made some people register in the Labor Offices in order to get the unemployed status and use the training opportunities offered in the SubComponent. 358 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Passive persons benefitted much less of the training. However, for both these groups the training was a “window to the world” and allowed them to see “better life”. The high effectiveness of training service was undoubtedly caused by the structure of agreement with the training agency, which made the amount of payment dependent on the effects. The training firms were well motivated to adjust the training contents to the needs of local labor markets and specific employers. It would happen that 100% of trainees found jobs. In this way private companies received well qualified workers and could develop faster. Temporary employment services The temporary employment sevice beneficiaries had a neutral attitude to the chances of finding a permanent employment. 73% of the beneficiaries thought that their chances neither increased nor decreased. Almost two out of three persons in that group did not take up work and did not start any business activity. The most often given reason for that was lack of the possibility of continuing the temporary work. But greater firms were more than smaller firms inclined to maintain the newly created jobs, even after the subsidy period had expired134. It does not mean, however, that temporary employment did not improve the quality of life of the beneficiaries. Important are the positive aspects of ending a prolonged inactivity. The temporary employment allowed to supplement the household budgets. Often persons were employed from families where both husband and wife had been jobless. Employment services The employment service beneficiaries much better estimated their chances of finding a job. More than half of them (55%) think that their chances of finding a job have increased. Percentage of those who found jobs thanks to the service is similar (57%). One third of them see their new job as permanent. Support centers and incubators of entrepreneurship Almost two thirds of the beneficiaries of entrepreneurship support centers think positively of their chances of finding a job. Every fourth beneficiary found employment and 27% of beneficiaries started their own business activity. At the same time more than 38% of that group declare that the centers did not help to find job or start own business. Contrary to other forms of support in Sub-Component B1, clear majority of the customers of entrepreneurship support centers (61%) are of the opinion that their new job or business activity are of permanent character. In the case of entrepreneurship support centers, synergy was observed with Component A, namely loan advisors worked also in the center offices. The interested businessmen received more complex service. The so developed business supporting network has been operating until now. In many cases the entrepreneurship incubators allowed companies to survive the most difficult period – starting their activities. In the opinion of gmina offices and labor offices, all the above mentioned services did not generate significant additional costs. Most often mentioned were typical office and administrative costs which had to be borne anyway in everyday office operations. 134 In one voivodship the percentage of temporary employment beneficiaries keeping their jobs after the end of service is estimated at 25%. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 359 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec 7.2.5 Economic effects of Sub-Component B2 Majority of teachers participating in the training activities (53%) did not bear any additional participation costs (e.g. traveling, hotel costs) or had their costs reimbursed. Average traveling cost in the teacher group was PLN 48 and cost of additional required fees PLN 140. To 41% of the participants, trainings bring additional benefits, e.g. more free time due to skills learned (70% of indications) or increased number of extra-curricural activities (37%). Besides, in the opinion of teachers, thanks to the training fewer pupils repeated class or left school before graduation (29%) and there were more pupils per one teacher (25%). 20% of the questioned noticed a connection between reduction of the number of educational establishments and the training realized in RDP. No significant relation was observed between acquiring new qualifications in the training and new income sources or change in the professional situation. In the opinion of 88% of teachers, their status as employees was raised. Very similar are indicators pertaining to the information technology training participants. In the school master group, percentage of persons with no additional costs of training participation was 52%. If necessary, additional traveling costs had to be borne of a value of about PLN 80 and additional required fees – PLN 160. Directors were more optimistic in expecting additional benefits from the training (58% of indications). They differ also in qualifying those benefits. 80% of them indicated more free time due to skills learned and 33% - increased number of extra-curricural activities. Also in this group no significant correlation was observed between participation in the training and new income sources or change in the professional situation. For 77% of the school masters, their status as employees was raised. The most often mentioned economic effects of the educational infrastructure investments are lower costs of heating, better natural environment condition and general appearance. Additional investment costs included the building and investor supervision costs, designing work, costs of documentation (construction and technical design), heating audit, costs of expert opinions and additional office work. These were not qualified costs and were borne by the gminas135. In the case of large investments also a feasibility study was prepared. Gminas will also have to bear operating costs of new objects. Construction and repairs of gymnasiums will extend the scope of curriculum in physical training as well as the school sport club activities. In several cases that will include new therapeutic exercise. 7.2.6 Economic effects of Sub-Component B3 The Sub-Component B3 activities stimulated optimization of the office structure. Initially it was connected with liquidation of ineffective work posts but in later stages new services were formed, e.g. business development service. More client-friendly structures are seen as one of the investment attracting elements. Officials participating in the SubComponent B3 could improve their qualifications and now they better perform their work. Also valuable training materials were prepared in the Sub-Component B3 and are now made available to the interested gminas as public property. 135 Estimates of additional unit costs accompanying the educational investments are PLN 10 000 – 20 000, with investments of an order of PLN several hundred thousand. It is worth noticing that these were fixed costs, so the project value was directly connected with the effectiveness of additional costs (the larger the project the smaller the fixed cost share). 360 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec An effect of projects connected with the National Training Program (NTP) was setting up new business units in every third gmina participating in the project. Most often 2 business units are created in a gmina. Figure 68. Number of business units to be registered as an effect of the NTP 19% 6% 47% 13% 7% none 2 business units 5 business units do not know / hard to say 6% 2% 1 business unit 4 business units more than 5 business units Source: Own research An effect of NTP are also new jobs in the off-farm sectors. Implementation of the NTP projects will create new jobs in the off-farm sectors in every second of the NTP participating gminas. Figure 69. Number of new jobs in the off-farm sectors as an effect of the NTP 11% 20% 44% 7% 11% none 11 - 20 new jobs 1 - 5 new jobs more than 20 new jobs 7% 6 - 10 new jobs do not know / hard to say Source: Own research Almost three quarters of the NTP participants declare that they have not incurred any additional costs in connection with the project. Half of them see benefits from NTP, such as: • Increased investment attractiveness of the gmina (63% of indications), • Increased tourist visits (48%), Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 361 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec • Greater personal income (41%), • Greater gmina income (67%). 7.2.7 Economic effects of Component C The most often mentioned effects of infrastructural investments in the RDP participating gminas is improved quality of life and health conditions. There is also a common opinion that in longer run the infrastructural investments are a necessary factor in attracting new investors and stimulating business activity of the inhabitants. Most often, however, the RDP investments attracted small investors. In many cases gminas required of the investment executors to employ local unemployed persons. New jobs were then created, although usually of a temporary character connected with the investment realization period136. It was most often physical work. Relatively many such jobs were generated by the water supply / sewage disposal investments, fewer by the road investments. A social effect of such work was that the seasonal workers regained their unemployment benefit rights and existence of many families was improved. In several cases, however, construction of automated sewage treatment plants caused liquidation of the existing work places. Very often the investment territories might be qualified in the territorial plans as investment or recreation areas. Then the interest in such territories usually increased considerably and so did the price. Each purchase / sale transaction caused a higher legal deed tax to be calculated. The gmina incomes were also increased from higher real estate taxes. Such situation occurred when new investors were brought to the developed areas. In some cases the sewage disposal investments allowed to cease to pay fees to the voivodship office for the economic use of natural environment, as soil degradation had been stopped. Realization of the investments did not generate any costs to the gminas. No additional office staff was employed to serve the investments and the costs involved were of an administrative character. An important element was the valuable knowledge gained by the gmina officials in preparing the investment finance support applications, which is now used e.g. in applying for the structural funds. Sometimes it was necessary to pay damages for losses caused by construction of pipelines on private premises. Very often the proper sewage disposal arrangements reduced the epidemiological hazards (some jaundice cases were even liquidated). Construction of bituminous pavement roads reduced the dust and exhaust gas content in the air and therefore improved the health conditions. Liquidation of septic tanks, improved tightness of settlement tanks raised the water cleanness class of the lakes, which evidently increased the tourist attractiveness of the gminas. In effect, tourists arrived which stimulated creation of agro-touristic farms. The existing hotels and camping fields, when connected to the sewage disposal network, reduced their costs and spared additional means for own development. Connecting to the sewage disposal network caused also more just sharing of the negative external effect costs. New users of the sewerage system had to start paying fees, but they ceased to pollute the environment (leaking septic tanks, illegal dumping to rivers etc.). 136 Depending on the type and value of investment, a dozen or so temporary jobs and several permanent jobs were created in a gmina, e.g. operating a plant, setting up service points etc. For instance, a PLN 270,000 worth water supply investment would generate 13 new jobs, including 10 temporary jobs. 362 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec The infrastructural investments had also a positive impact on the general appearance. An example are sidewalks built with the water supply system. Perception of the Component C economic effects may be different from the point of view of the local government, household representatives and business people. Therefore, a more detailed analysis of the economic effects is divided according to those three groups of respondents. Economic effects in the opinion of local government representatives In the opinion of 82% of the representatives of RDP participating gminas, the water supply investments caused improvement of the living standard of population. Only one in ten of them indicated new agro-touristic hotel places as an effect of the investment. In 6% of cases new recreation or touristic areas were allotted after the investment. Relatively small was the impact of water supply investments on attracting new investors. 38% of the gmina representatives indicated such effect. The most important effect of the sewerage system construction was in the opinion of gmina representatives an improvement of the living conditions of rural population. Such effect was indicated by 90% of respondents. One third of them indicated greater interest on the part of investors. Every fourth respondent indicates creation of new hotel places as an effect of the investment. Construction of the water supply system was very important for development of the gminas. The most often mentioned aspects of that development were improvement of the natural environment, better sanitary conditions and an incentive for business activity. As the road construction is concerned, the most important and immediately observed effects were improved road safety (more than 61% of indications) and fewer vehicle defects (80%). Transport of schoolchildren improved considerably (90%), and also access to fields and neighbouring villages (80%). Mentioned also were better safety (connected with e.g. shorter waiting time for ambulance – 100% of indications) and better business activity conditions (90%). Increased interest on the part of investors is noticed by 40% of respondents. All the above mentioned factors contribute to a considerable improvement of the conditions of life. Such an effect was mentioned by more than 90% of the gmina representatives. Economic effects in the opinion of household representatives Also in the opinion of household representatives, the infrastructural investments considerably improved the living conditions in the respective gminas. Also improved are the general appearance of villages (21% of indications) and the health situation (13%). Every tenth village inhabitant indicated an increased investment attractiveness of the gmina where he/she lived. Relatively few indications mentioned correlation between the RDP investments and improved farming conditions, own business activity or social activity. Positive answers to a question concerning new sources of income, improved financial or work situation are within a statistical error. The sewerage system investments, in the opinion of household representatives, were not connected with creation of new jobs. Only 12% of respondents can indicate people who found employment in connection with such investment. Connection of a household to the sewerage network did not cause repairs of other compartments (e.g. bathrooms). Additional jobs were not generated in this way as repairs, if any, were performed by the household members. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 363 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Construction of new roads was clearly connected with the improvement of the conditions of life (almost 40% of positive indications and 4% negative opinions). The most important improvement factors were: easier transport of schoolchildren, better road safety and shorter access time to neighbouring villages. Road construction had little effect on the traffic safety and the number of vehicle defects. Economic effects in the opinion of business people In the group of investigated business units, more than two thirds were of the opinion that the infrastructural investments had a great impact on the long term business activity. In that sense, the most important investments were water supply networks and good roads. However, in shorter time the realization of infrastructural investments was not connected with additional production or service capacity. The investments were also neutral as regards the costs of functioning of business units and the employment decisions. As a rule, no new sources of income appeared with those investments. Also there was little connection between a company location decision and the investment. However, the infrastructural investments were of great importance for further business development and also for the investment attractiveness of a gmina, improvement of living conditions and increased business activity of the inhabitants. In the opinion of business people, infrastructural investments caused price increases of arable land and building lots as well as extension of investment areas. What is important, in their opinion the infrastructural investment realizing gminas might expect new investors. 7.3 Comparative (benchmarking) analysis of project effectiveness In order to evaluate the Project effectiveness, comparison was made of effects and expenditures and then the obtained indicators were compared with available data from other programs or, for the construction work, with tender data. As the reports from analysed programs not always contained all the data of interest to the consultants, different projects will be a reference point for the individual indicators. In the projects where data were given in EURO or USD, expenditures and costs were converted at a mean exchange rate of 4.09 PLN/ EURO and 4.01 PLN/ USD. Those exchange rates are mean values for the years 2000 – 2004. Assuming a mean value of exchange rate is a simplification but from the available data it was not possible to evaluate the expenditure distribution over time and assign the then valid exchange rates. Among the realized projects is the ASAL project from the years 1994 – 1997. As eight years have passed since the completion of ASAL and inflation from 1997 was about 70%, then as an exception for that project a direct recalculation was made of the project expenditures into the 2005 prices. The expenditures had to be recalculated in order to ensure comparability of data. One of the sources of comparisons was the Bulletin of Public Procurements. Selected from it were a number of biddings from the years 2001-2004 in the water supply, sewerage and road construction branches, wherever the financial and quantitative information was available and the character of work was similar to that of RDP. It has to be remembered that the comparisons given below are of illustrative character and may be inaccurate due to specific features of each of the compared projects and their internal non-uniformity. These factors may make the effects difficult to compare. 364 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Attention has to be drawn to the fact that in the infrastructural investments the basic objective is to improve the technical standard of living of the population and not to generate new jobs, therefore the cost of creating a new job as a quotient of the investment expenditures and the number of new jobs created will be in this case exceptionally high and incomparable with e.g. the micro-loan or training Component. 7.3.1 Microloans Due to lack of sufficiently reliable data, it was not possible to analyse the Component A effects against results of other projects. Nevertheless, the cost of creating one new job (PLN 4 268), calculated as the value of Component A minus the value of repaid loans and divided by the number of created new jobs must be considered low. However, data were available to compare the organizational efficiency in granting RDP micro-loans against the efficiency in other projects and institutions. The average sum of loan granted in RDP was 43% higher than the average of seven projects (institutions), including RDP. The RDP loans were relatively high, therefore their importance for starting (or maintaining) business activity and creating jobs was considerable. Figure 70. Average value of a loan granted within the analysed projects and by financial institutions (thous. of PLN) 20 16 12 17,1 8 14,3 12,6 4 6,4 5,7 3,6 1,8 0 RDP 110 selected 12 micro-loan institutions in micro-loan Eastern institutions European countries Fundusz mikro Inicjatywa mikro loanable funds (72 loanable funds in Poland) MicroCredit Fund Source: Project realization reports, data of financial institutions, own analyses Average cost of a loan was relatively high. However, it should be noticed that the loan value was relatively high and percentage of lost loans exceptionally low. Besides, the cost of loan is comparable with that in the Polish banks. It is, however, much higher than the cost of loans granted by the Eastern Europe funds and the more so with loans granted by funds in other parts of the world where labor costs are much lower. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 365 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Figure 71. Average cost of granting a loan per one borrower within the analyzed projects and by financial institutions (thous. of PLN) 1,5 1,2 0,9 1,31 0,6 1,25 0,92 0,3 0,36 0,0 RDP 110 selected micro-loan institutions 12 micro-loan institutions in Eastern European countries Bank X (Poland) Source: Project realization reports, data of financial institutions, own analyses Efficiency of the credit officers (loan advisors) is on a good level among the investigated institutions (projects). Higher efficiency was achieved only with much lower loan values. Figure 72. Efficiency of credit officers in the analyzed projects and financial institutions (number of borrowers/ credit officer) 350 280 210 326 308 140 194 171 70 90 67 0 RDP 110 selected micro-loan institutions 12 micro-loan institutions in Eastern European countries Bank X (Poland) Bank Y (Poland) BOSPO Source: Project realization reports, data of financial institutions, own analyses No reliable data were available to compare the cost of creating one new job137. It may be assumed, however, that the cost of creating one new job in the Project, i.e. PLN 4,300, was low. 137 That cost is a value of Component A minus the sum of loans granted and divided by the number of new jobs, according to MIS. 366 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec 7.3.2 Training As the available information on training and its effectiveness is scarce, it was only possible to compare the average cost of training and percentage of people who took up work after training. The comparison included, apart from RDP, also data from the SAPARD project, PHARE – Inicjatywa project and from the Ministry of Economy and Labor (MEL) on the labor market and unemployment prevention programs in 2003, published in the “Report on the labor market and social security”. The above mentioned data show that the average training cost in RDP was the highest among the compared projects, but we do not have accurate information on the type of training in SAPARD and in the government programs. Cost of one training day may be significantly different depending on the subject, number of trainees and many other factors. Therefore, the results of comparison should be treated with great caution. Figure 73. Average cost of training per person (PLN/ person) 1 000 800 600 980 863 400 794 540 200 0 RDP SAPARD Government Programs in 2003 Average Source: Project realization reports, data of financial institutions, own analyses Percentage of persons who started work after training is relatively high, although no exact data on the duration of such employment are available. Conclusions may be drawn from field investigations that for 2/3 of those trained the effect was not durable. For comparison, in the “Inicjatywa” project only ¼ of those initially employed kept the employment. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 367 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Figure 74. Percentage of persons who started work after training 42% 35% 28% 21% 14% 40,2% 31,8% 31,1% 24,0% 7% 0% RDP Program PHARE Inicjatywa Government Programs in 2003 Average Source: Project realization reports, data of financial institutions, own analyses 7.3.3 Construction of water supply networks For the comparison of effectiveness of spending the water supply investment means, information was used from the SAPARD, FDL, ASAL projects as well as bidding data for the water supply system construction work, published in the Bulletin of Public Procurements. As mentioned earlier, the ASAL project financial data were indexed by the inflation rate since 1997, i.e. 70%. Average construction cost of one kilometer of the water supply system is lower than in the compared projects and only slightly higher than a market cost of that type of work performed on the tenders quoted in the Bulletin of Public Procurements. Figure 75. Average construction cost of 1 kilometer of water supply system (thous. of PLN/km). 75 60 45 75 30 60 56 46 43 15 0 RDP SAPARD FDL Open public bidding 2003-2004 Średnia Note: Network construction costs in RDP do not include the water conditioning station expenditures. Source: Project realization reports, Bulletin of Public Procurements, own analyses 368 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Average cost of one connection is very low. We do not have more precise information on the comparability, i.e. whether in the compared projects any other costs were included, e.g. the water conditioning station. Figure 76. Average cost of one water supply connection (thous. of PLN/ conn.) 7,5 6,0 4,5 6,5 3,0 6,0 6,0 6,0 FDL ASAL (estimates) Average 3,5 1,5 0,0 RDP SAPARD Note: Connection costs in RDP do not include the water conditioning station expenditures. Source: Project realization reports, own analyses Average cost of creating one new job is also very low and as in the case of network construction, we do not have information whether in the compared projects any other costs were included, e.g. the water conditioning station. Figure 77. Average cost of creating one new job in the construction of water supply system (thous. of PLN/ one new job) 140 105 70 129 35 76 75 ASAL (estimates) Average 47 0 RDP FDL Note: New job creation costs in RDP do not include the water conditioning station expenditures. Source: Project realization reports, own analyses 7.3.4 Construction of the sewage disposal networks For the comparison of effectiveness of spending the sewage disposal network investment means, information was used from the SAPARD, FDL, ASAL programs as well as bidding data for the water supply system construction work, published in the Bulletin of Public Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 369 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Procurements. As in the analysis of water supply system investment effects, the ASAL program financial data were indexed by the inflation rate since 1997, i.e. 70%. Average construction cost of one kilometer of the sewage disposal network is lower than the market values published in the Bulletin of Public Procurements from the actual contracts. The sewage disposal network construction cost does not include the sewage treatment plant costs. Figure 78. Average construction cost of one kilometer of sewage disposal network (thous. of PLN/ km) 350 280 210 333 140 70 274 205 169 189 0 RDP SAPARD FDL Open public Average bidding 2001 2004 Note: Network construction costs in RDP do not include the sewage treatment plant expenditures. Source: Project realization reports, Bulletin of Public Procurements, own analyses As the construction cost of one kilometer of the sewage disposal network, also the cost of one connection is low. Figure 79. Average cost of one sewage disposal connection (thous. of PLN/ conn.) 20 16 12 17,5 8 14,3 13,3 4 8,3 7,7 0 RDP SAPARD FDL ASAL (estimates) Average Note: Connection costs in RDP do not include the sewage treatment plant expenditures. Source: Project realization reports, own analyses 370 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Cost of creating one new job in the construction of sewage disposal network, without the sewage treatment plant expenditures, was PLN 137,000. With the sewage treatment plant expenditures, the cost of creating one new job in RDP would be PLN 172,000. Figure 80. Average cost of creating one new job in the construction of sewage disposal network (thous. of PLN/ one new job) 280 210 140 254 209 137 70 77 0 RDP FDL ASAL (estimates) Average Note: New job creation costs in RDP do not include the sewage treatment plant expenditures. Source: Project realization reports, own analyses 7.3.5 Road repairs For the comparison of effectiveness of spending the road investment means, information was used from the SAPARD, FDL program as well as bidding data for the road construction work, published in the Bulletin of Public Procurements. Average road repair cost financed from the RDP was 11% higher than the bidding values published in the Bulletin of Public Procurements. Figure 81. Average repair cost of one kilometer of road (thous. of PLN/ km) 350 280 210 140 307 349 276 333 70 0 RDP SAPARD Open public bidding 2003 - 2004 Average Source: Project realization reports, Bulletin of Public Procurements, own analyses Average cost of creating one new job in the road works in RDP was PLN 55,000 and was considerably lower than in FDL. We have no knowledge of the type of contracts Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 371 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec concluded by the FDL units and in particular of the technologies used, which may significantly influence the cost of one workplace. Figure 82. Average cost of creating one new job in the road construction (thous. of PLN/ one new job) 100 80 60 90 40 73 55 20 0 RDP FDL Average Source: Project realization reports, own analyses 7.4 Statistical analysis 7.4.1 Subject of investigation Subject of the statistical analysis was investigating the cause-effect relation between the Project intensity indicators in the individual territorial units (powiats and gminas) and the social and economic effects corresponding to the basic Project result indicators. For the analysis in question, the following indicators were assumed as a measure of the Project intensity in its individual Components (outputs): • Number of loans granted in Component A, • Number of beneficiaries in Component B1, • Value of the financial means spent in Component C. For this analysis, the following values were assumed as indicators of the Project outcomes: • Change in the number of registered private business units – in Components A, B1 and C for powiats and additionally in Components A and C for gminas; • Change in the value of the local government income from the share in personal income tax138, in Components A, B1 and C for powiats and additionally in Components A and C for gminas; 138 An additional simplification was used in the income of powiats as it appeared difficult to obtain data on the budget income from that source; according to the regulations, the gmina share in personal income tax is 27.6% of taxes collected on the gmina territory, the powiat share is 1% of 372 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec • Change in the number of private sector employees – in Components A, B1 and C for powiats and additionally in Components A and C for gminas. The Component B1 analyses at the gmina level were impossible due to the method the data on beneficiaries were recorded – they were available at the powiat level only. 7.4.2 Methodology of the analysis In order to determine the cause-effect relation between the Project intensity indicators in the individual territorial units and the social and economic effects of the Project (outputs and outcomes), the correlation between those values was analysed by calculating the linear correlation coefficient defined as: By calculating the correlation coefficient, with a relatively large group of the territorial units it was possible to eliminate partly the impact of disturbing factors influencing the changes in employment, business activity or wealth in the individual gminas or powiats. The use of individual result indicators is justified as follows: • Increased number of private sector jobs is most coherent with the assumed Project objective: „number of created off-farm new jobs in gminas participating in the project”; by neglecting the increase in public sector jobs we wanted to eliminate the „bureaucratic” jobs as it is mainly there that the public sector generates new jobs these days; • Increased share of the local government units (gminas and powiats) in the personal income tax is roughly proportional to the off-farm incomes of the population (the agricultural activity does not generate that tax); the not full proportionality is caused by a tax free sum, progressive tax scale and different cost of drawing the income, usually higher in the individual business activity (this problem is discussed later in the report); nevertheless, this is the best available and most reliable indicator of the offfarm incomes; • Increase in the number of business units, according to the REGON records, is also disturbed to some extent because it includes large companies and individual business activity; additionally, the business activity suspending units are not deleted from those records; but with reasonable accuracy the increase in REGON register list may be treated as increase in the number of new individual business units. It should be mentioned that these data are available and identically defined in the GUS reports for the years 2000, 2001, 2002 and 2003. We are aware that the proposed method of analysis may only show relation between intensity of the Project in a given territorial unit and short term effects visible before the taxes collected on the powiat territory, so the powiat income was estimated by summing up incomes of the respective (powiat forming) gminas and dividing the sum by 27.6. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 373 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec end of 2003. The long term effects, the main target of the Project, will be measurable when a similar analysis is performed within the next several years. However, each of the investigated Components A, B1 and C had also short term effects of a significant social and economic importance. Such effects include: • Creating new business units (or preventing the existing ones from insolvency) and employing new workers shortly after receiving a micro-loan by them. • Finding new job by the Component B1 beneficiaries soon after taking part in the Project, • Setting up businesses, creating new jobs and generating additional income in gminas and powiats when the infrastructural investments have been carried out. Evidently the long term effects, such as increased business attractiveness of the area with improved infrastructure or the multiplier effect of new businesses, jobs and increased wealth, will not be noticed or will be noticed only to a minimum degree. There is no method of measuring these relations now. Conclusions may be drawn only indirectly from the experience of other similar projects, although such conclusions might be highly speculative, taking into account the specificity of RDP and exceptional circumstances of the accession to the European Union. At the time of this analysis no statistical data for 2004 were available. Therefore statistical data for the years 2000-2003 were analysed. In order to achieve compatibility of data and effects, expenditures until the end of 2003 were taken for Component C. Unfortunately, it was not possible to separate the loans granted in Component A till the end of 2003 or persons trained in Component B1 till the end of 2003, so for those Components data were assumed from the whole Project realization period. The number of gminas in the analysis is different for different Components – it corresponds to the number of gminas that the beneficiaries came from. Also the number of powiats in the analysis is different for different Components, but it is not equal to the number of powiats the beneficiaries came from, because: • Among the powiats the beneficiaries came from were (sometimes) municipal districts; those towns, often with significant increase of employment (active businesses) and with great changes in budgets would disturb the analysis and therefore they were excluded from the analysis; • Among the powiats the beneficiaries came from were powiats which changed their boundaries in the analysed period (there were 38 such powiats in the country); that caused changes in the level of employment (and changes to the number of new jobs), new business units and budget income; that would also disturb the analysis and therefore those powiats were excluded from the analysis; The analysis took into account those powiats (with the exclusion of the above mentioned) which were themselves Project beneficiaries or their inhabitants (or gminas) were beneficiaries, and a powiat was included in the analysis even if just one gmina was a beneficiary. 374 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec Table 119. Number of gminas and powiats included in the analysis, depending on the Component Component A Component B1 Component C Number of gminas 538 0 481 Number of powiats 84 118 215 Source: own analyses. In the analysed gminas 4605 loans were granted and 4040 loans in the powiats. The number of loans in powiats is lower than the number of loans in gminas as some powiats were excluded from the analysis. In the analysed gminas where Component A was realized 54,800 new business units were created, 10,500 jobs were liquidated and the amount of budget incomes from personal income tax was reduced by PLN 64.4 million. Table 120. Changes in the number of business units, incomes and employment in gminas and powiats taking part in Component A in the years 2000 – 2003 Gminas Powiats Number of loans granted 4 605 4 040 Change in the number of business units 54 859 54 862 Change in the number of business units (%) 11,5% 13,4% Change in incomes from PIT (millions of PLN) -64,4 -2,9 Change in incomes from PIT (%) -5,7% 8,0% -10 476 -1 401 -1,9% -0,3% Change in employment in private sector Change in employment in private sector (%) Source: own analyses, RDP MIS, GUS data from the Poland 2001, 2002, 2003, 2004 blocks. In the analysed powiats there were 62,000 beneficiaries in Component B1. With such a large number of trained persons, the private sector employment was reduced by 4,700 persons, which was 0.7% of the number of employed in 2000. That reduction was caused by the increasing economic recession. Table 121. Changes in the number of business units, incomes and employment in powiats taking part in Sub-Component B1 in the years 2000 – 2003 Powiats Number of trained persons 62 019 Change in the number of business units 82 974 Change in the number of business units (%) 14,8% Change in incomes from PIT (millions of PLN) -4,2 Change in incomes from PIT (%) -8,3% Change in employment in private sector -4 696 Change in employment in private sector (%) -0,7% Source: own analyses, RDP MIS, GUS data from the Poland 2001, 2002, 2003, 2004 blocks. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 375 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec In the analysed gminas the Component C assigned means amounted to PLN 366.5 million and to PLN 432.2 million in the powiats (these means do not add up as most of the analysed gminas were part of the analysed powiats). The powiat means include the gmina means and money spent on projects where beneficiaries were powiats as such. Table 122. Changes in the number of business units, incomes and employment in gminas and powiats taking part in Component C in the years 2000 – 2003 Gminas Powiats Value of assigned means (millions of PLN) 366,5 432,2 Change in the number of business units 42 295 90 973 Change in the number of business units (%) 13,7% 14,1% Change in incomes from PIT (millions of PLN) -50,1 -4,0 Change in incomes from PIT (%) -7,0% -7,2% Change in employment in private sector 3 677 4 667 Change in employment in private sector (%) 1,0% 0,6% Source: own analyses The following correlation coefficients were obtained from the analysis of effects and expenditures (number of loans, number of trained persons, money invested): Table 123. Coefficients of correlation between participation of beneficiaries in Components A, B1 and C and the analysed statistical data Increase of income from PIT Increase of private sector employment Increase of the number of business units A – number of micro-loans in gminas -0,085 0,048 0,006 A – number of micro-loans in powiats -0,146 0,180 -0,032 B1– beneficiaries in powiats -0,033 -0,159 0,171 C – investment expenditures in gminas 0,019 0,098 0,190 C – investment expenditures in powiats -0,234 0,051 0,398 Component / Indicator Source: own analyses 7.4.3 Comments on results and conclusions The calculated correlation coefficients indicate that there is no noticeable relation between the Project intensity on a given territory and the short term increase of the offfarm incomes and employment in private sector. The reasons of that are the following: 1. The investment expenditures in the project are too low per a territorial unit (gmina or powiat) to change significantly its social and economic situation – particularly the level of employment and wealth. 2. The analysed period (2000-2003), as the most recent GUS data are from the year 2003, does not allow to take into account full effects of the project, i.e. the year 2004 376 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec and later. It may be expected that long term effects evaluated by the same method will be statistically more significant. 3. In the analysed years, many circumstances cumulated influencing the analysed indicators, namely: a. economic slowdown in the whole country, b. significant increase of unemployment in the country, c. multitude of the accession programs influencing the same social groups and territorial units, d. changes in the wage and tax regulations (particularly the so called pension reform), causing sharp increase of „self-employment” after the year 2000. The correlation coefficient determination method eliminates part of the cause-effect relation disturbances, provided they are of a statistical character and the analysed sample is very large. But not all disturbing factors may be eliminated by calculating correlation.. 4. Some of the newly employed or new business units might take jobs and market away from others. For that reason the absolute increase might have been smaller than the number of persons who found jobs through the Project, the more so that those were years of the economic slowdown. But even in such cases there may be some advantage as worse thing is driven out by better. The personal income tax is not fully proportional to the off-farm income. This is due to the fact that the single person business units may temporarily pay lower taxes than employees with a similar income. Therefore the correlation between the increased number of such units and the tax may be even slightly negative at the beginning – this can be seen in our analysis. Noticeable but also weak correlation may be seen between the infrastructural investment expenditures and the increased number of business units. It may be assumed that execution of significant infrastructural investments caused boom in the respective construction branches. As the work and deliveries were contracted through biddings, a condition of participating in them was „formal” employment or business activity and in effect new business units were set up also by those working so far on the “black market”. Correlation between infrastructural investment expenditures and the number of new business units is greater for powiats than for gminas. That may be due to the fact that the investment work and deliveries in one gmina usually engage firms and people from the neighbouring or more distant gminas and business units are often registered in the powiat towns and not in their own gminas. Summing it up, direct impact of the Project on the increase of employment, incomes and wealth in the years up to 2003 was hardly noticeable. It cannot be excluded, however, that such positive effects will be seen in the future. An empirical proof of that will be found by repeating the analysis in the next years. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 377 Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec 7.5 Summary and conclusions The scale of fiscal benefits due to creating 10 265 new jobs in the RDP, calculated with an updated at the end of 2003 methodology and macroeconomic data, amounted to a sum from PLN 47.03 million to PLN 121.66 million, depending on the wage a newly employed person received. This benefit is only one of many. The complexity of RDP is connected with a spectrum of economic effects. Not all the relations can be caught within a few months after the end of the project. From the direct investigations it may be found that relatively soon have appeared effects of the granted micro-loans and subsidies as well as of some labor market dedicated services. Full effects of the infrastructural investments and effects of the increased institutional potential of local governments will take some more time. The greatest delay between the expenditures and sustained effects will be in the educational activities. Undoubtedly the RDP has created foundations of faster economic development of rural areas through: • preparing investment sites, • better effectiveness of administration, • activisation of the unemployed, • development of a network of business supporting institutions. That development will probably be strengthened through the support funds, which have now become much easier obtainable to the RDP participating gminas. The quality of living conditions of the rural population from the RDP covered territory has already visibly improved and there are first symptoms of practical investment activity of business units there. Also meaningful is the positive impact of the project on the local labor markets. Many unemployed have been awakened from apathy and inactivity after many years of joblessness. Even short term actions consisting in temporary employment had a very positive impact on the family budgets. The costs of participating in the project on the part of beneficiaries and the RDP implementing gminas were low. They were usually limited to some administrative and traveling costs and devoting some spare time to it. However, these conclusions, drawn from direct investigations, are of qualitative character and do not allow to evaluate the scale of effects. Comparing with other similar projects (within individual Components as there were no other analogous projects), RDP looks well, both in terms of cost effectiveness and material efficiency. Moreover, comparison with the open tender indicators (taken from the Bulletin of Public Procurements), close to the market values, also showed similarity to the RDP results. A cautious conclusion may be drawn (taking the significant differences of the projects into account) that the public money spending mechanisms in RDP were effective and there was no money squandering in statistical terms (there might have been, though, isolated cases of that). Analysis of the correlation between the Project intensity indicators within the Components A, B1 and C (outputs) in the individual gminas and powiats and the main Project result indicators (outcomes), i.e. increased employment, incomes and wealth gives in most cases close to zero results. That leads to a conclusion that the impact of RDP on the economy of participating territorial units was hardly noticeable in the years up to 2003. This is a statistical conclusion and limited to the short term effects only. One of the main reasons of weak macroeconomic effects of RDP was its broad scope and therefore the 378 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Analysis of the social and economic effects Jarosław Malec concentration of expenditures in a single gmina or powiat was too small to change the economic situation in a radical way. It is possible that similar analysis will bring more visible long term impact when the multiplier effects of the Project become evident. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 379 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. 8 Evaluation of the institutional and organizational solutions and the quality of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 381 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. 8.1 Introduction The Rural Development Project was formally started on 25 July 2000 when the Polish government and the International Bank for Reconstruction and Development signed a credit agreement for its realization. Formal closing of the Project, originally scheduled for 31 January 2004, is to take place on 30 June 2005.139 This evaluation of the Project management system is based on the information from three main sources: (a) from the Project organizational structure and obligatory procedure describing documents, (b) from records of a report character documenting the Project realization and results of control operations in the process, and (c) from the results of questionnaire investigation of more than thirty groups of respondents pronouncing on the Project course and results. Among the documents confirming formally the applied procedures, particularly interesting are the Project Appraisal Document (PAD) and the Operational Manual together with its annexes. They describe the tasks and mutual relations of all the important institutions participating in the Project. Remarkably much room is devoted to the institutions specially set up for the realization of RDP, i.e. the National and the Regional Steering Committees, Project Coordination Unit and Voivodship Project Implementation Units. This chapter presents also some of the internal use documents prepared by the institutions participating in the Project. Typically, those documents give details of the basic project documents. Characteristic of the Rural Development Project is the very rich reporting documentation due to its complex organizational structure – both vertical and horizontal. It reflects also the high priority that the monitoring function has been given in the Project. In the information of a reporting character the following may be distinguished: constantly updated MIS (Management Information System) and MIS B1 databases, periodical reports, prepared by the Project Coordination Unit and presented to the National Steering Committee and the World Bank and devoted to the realization of the whole Project, and also reports on the individual Components prepared by the respective participating units. Specific place in the Project monitoring document list takes the Supreme Chamber of Control (NIK) report. Published in August 2003, the report documents the results of control carried out from January to March 2003. It summarizes the then far from satisfactory results of the Project and formulates critical opinions on the Project organization as the main cause of the difficulties. It presents also to the Project managing institutions postulates aimed at avoiding similar difficulties in the future. In a natural way, that document constitutes a reference point for the authors of this final appraisal. The information, opinions and suggestions contained in it will be referred to and commented upon further in this chapter. Most of the research tools used in the questionnaire investigation performed specially for the final appraisal, contained questions on such problems as: activities of the Project realizing institutions, level of services provided to persons and institutions taking advantage of the Project offers or respondent opinions on the weaknesses and strengths of the Project organization and desired changes. 139 382 The Project final appraisal study was prepared between December 2004 and May 2005. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. The questionnaire investigation results have been made use of in this chapter in two ways: • two subchapters discuss collective opinions of respondent groups which include Project beneficiaries, local government representatives (depending on the Component – in a double role of beneficiaries or Project organizers at the lowest level) and the RDP implementing institutions at the national and regional level; • subchapter presenting an evaluation of usefulness of the RDP solutions for future project management conception. 8.2 The conception of Project management Conception of the scope of activities and an appropriate organizational form of the Rural Development Project was created during joint work of the World Bank experts and the Polish government representatives. The worked out terms and provisions were first presented in March 2000 in two almost simultaneously published documents – the Project Appraisal Document (PAD) prepared by the World Bank experts and the “Operational Manual” which was then made an appendix to the credit agreement concluded between the Polish government and the World Bank. It is worth noticing that according to the Supreme Chamber of Control (NIK) report „it was permitted that the objectives and method of using the Project credit were determined by the World Bank and not by the Polish side (…). The Project Appraisal Document prepared for the World Bank by a team of its experts had not taken into account all the remarks put forward by the Ministry of Agriculture and Rural Development and had not obtained a formal acceptance of the borrower but it was a basis for preparing the credit agreement.”140 As the Project Appraisal Document was devoted first of all to the presentation of assumptions and general form of the Project, the Manual was concentrated on the practical aspects of its realization, such as description of the planned actions, proposals of subdivision of the tasks between the parties involved or of working procedures, together with proposed contents of the essential documents. The conception presented in both those documents took into account some specific features of the Project, making it organizationally very complex and at the same time a novel undertaking: • High priority was given to obtaining a strong involvement of the representatives of local and regional governments („Premise of active participation”). The use of active participation approach is of key importance to the whole regional development conception. The local government should be included into the decision making process of fund allocation, in other words support without involvement of the beneficiaries carries a risk of longer term inefficiency and may lead to an excessive dependence. Besides, the use of an active participation approach is required by the present process of decentralization of the administrative functions and financing in Poland, where the voivodship governments have an obligation to present to the national government a proposed regional development plan. Having that in mind, the RDP has been designed with the bottom-up approach. The applications for financial 140 „Information on the results of control of the sources and use of means of the Poland Rural Development Project” Supreme Chamber of Control, Warsaw, August 2003. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 383 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. support within the Project were submitted by the interested gminas, powiats and voivodships. Those applications reflected the priorities determined in the local development projects and as such they were approved by the local governments. Regional Steering Committees were set up with participation of representatives of all social groups and their task was to work out a ranking list of the presented proposals. • Some social analyses were performed in parallel in order to adjust best the Project structure to the local needs and priorities.”141 • Working procedures, prepared by the World Bank, different from the legal solutions in force in Poland, were proposed as obligatory in the Project. • The activities planned in five Components and Sub-Components of the Project were a subject of interest to five ministries; in each of the ministries, except the Ministry of Finance, groups of specialists were set up to deal with the Project and in particular the Component or Sub-Component of interest to a ministry (Project Teams), • A substantial part of the work was to be ordered with external executors selected by open bidding. The Project planners, attempting to organize each of the Project Components and SubComponents in a most effective way, devised and presented in the project documents five separate and basically different organization plans. The organizations listed in the table below took part in realization of each of the Components in a network of relations specific to that Component. Key factors having a decisive impact on the degree of complexity of the Component organization plans were decisions on which selected tasks would be ordered with commercial service companies and which institutions would be chosen to supervise the realization of those tasks. Those decisions, together with estimated values of the planned contracts, caused, as a consequence, decisions on the selection of: (a) procedure – from among the available World Bank procedures, applicable to the bidding in question, (b) contract realization monitoring method, (c) work accounting procedures and methods of payment. The need of coordination of the activities of many national and regional institutions was a reason of setting up a number of new organizations as decision making and administrative bodies, designed from scratch for serving the Project. On the national level this is the National Steering Committee (a decision making body) and the Project Coordination Unit (administrative support). Their regional level counterparts are the Regional Steering Committees (decisions) and the Voivodship Project Implementation Units (administration). In effect, the overall Project (and its Components) organization plan presented in the Project documents has several specific features: • 141 384 It was prepared with an assumption that the institutions responsible for Project realization would use their own earlier used procedures, in order to get them implemented quickly, PAD, p. 11. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. • Key national and regional level Project institutions just started their activity or did not exist yet at the time of beginning of implementation, • The Project realization method – in its target version – was to be an outcome of four complementary elements: (1) the World Bank imposed methods and procedures, (2) organizational solutions and resources of the Polish national, regional (voivodship) and local (powiat, gmina) level offices, (3) work practice applied by the specially set up Project serving units and (4) methods of work of the subcontractors performing tasks defined in the respective contracts; details are presented in the table below. Table 124. Organizations involved in realization of RDP Level of involvement National Type of organization World Bank World Bank Office in Poland Polish permanent institutions Ministry of Agriculture and Rural Development Ministry of Economy and Labor142 Polish special institutions National Steering Committee Ministry of Finance Project Coordination Unit Ministry of National Education and Sport Project Teams in Ministries Ministry of Interior and Public Administration Subcontractors National Economy Bank Consultants (technical assistance) Service providers (e.g. training) Suppliers Agency for Reconstruction and Modernization of Agriculture Regional Marszalek Offices Voivodship Labor Offices Voivodship Education Supervision Offices Regional Branches of the Agency for Reconstruction and Modernization of Agriculture Local Regional Steering Committees Voivodship Project Implementation Units (VPIU) Consultants Service providers Suppliers Powiat Labor Offices Consultants Powiat Starosta Offices Service providers Gmina Offices Suppliers Source: own study Rural Development Project is therefore an organizational experiment based on complex assumptions, broadly conceived, having complicated implementation process and not unequivocal results. 142 As of April 2005. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 385 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. 8.3 Institutions and functions of the Rural Development Project in the light of program documents National Steering Committee was established to „(…) distribute the Project means among the voivodships, supervise the Project implementation, analyse the annual audits, solve all the problems arising in the coordination process and approve amendments to the Operational Manual.”143 Members of the Committee are representatives of the ministries participating in the Project and two representatives of the Convent of Marszaleks. Minister of Agriculture, as the NSC Chairman, was supposed to invite representatives of the Ministry of Environment, European Integration Committee, Chairmen of the Rural Gmina Association and Powiat Association to take part in the NSC activities..”144 Tasks of the Project Coordination Unit (PCU) are described as follows: „The National Steering Committee will be supported by the Project Coordination Unit located in the structure of the Foundation for the Assistance Programs for Agriculture (FAPA), an independent agency reporting to the Minister of Agriculture and Rural Development (…).” PCU is „responsible for overall coordination of the Project and will also fulfill a function of secretariat to the National Steering Committee. The Project Coordination Unit will receive periodical progress reports from VPIU and from program teams in the respective ministries. It will also prepare summary reports for the government, Bank and in appropriate scope for other co-financing organizations. The Project Coordination Unit will be responsible for organizing the Project initiating workshops and for ensuring support for all the Project management levels.”145 Reasons for setting up Program Teams in the ministries have been presented in the Operational Manual: „Each ministry participating in the Project will nominate a group of persons or authorize an external unit to perform the monitoring functions. Minister of Agriculture and Rural Development will sign Implementation Agreements with each minister responsible for implementation of a Component. The agreements will determine obligations of the parties and the procedure of financial means reallocation from one ministry to another if it turns out during Project realization that the necessary scope of work in a task is smaller than planned.”146 The Regional Steering Committees are characterized in the Operational Manual as follows: „The Regional Steering Committees are formed in each voivodship included in the Project. The Committees will perform the following functions: • Approval of projects prepared by gminas and appraised by the Voivodship Project Implementation Units, establishing project hierarchy based on the project importance to the regional development, gmina financial contribution and the economies of scale. • Concluding agreements with the National Steering Committee for assigning the Project financial means to the participating voivodships. The agreements will be based on the financial means allocation criteria formulated by the National Steering Committee (e.g. percentage of population living in rural areas) and will be signed by 143 Project Appraisal Document, 21 March 2000, p. 17. Project Appraisal Document, 21 March 2000, p. 17. 145 Ibidem, p. 18. 146 Operational Manual, 9 March 2000, p. 8. 144 386 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Marszalek as Chairman of Regional Steering Committee and Minister of Agriculture and Rural Development as Chairman of the National Steering Committee. The Regional Steering Committee will work under the chairmanship of Marszalek of voivodship and will be composed of representatives of local governments, education, chambers of commerce, agriculture, industry, trade and non-governmental organizations.”147 It as assumed that „functions of the Voivodship Project Implementation Units will include: • Project administration on the local level, • Assistance to beneficiaries in preparing project applications, • Reviews of project applications and qualifying the projects, against qualification criteria, for inclusion to project portfolio subject to final approval by the Regional Steering Committee, • Receiving notices from the Regional Steering Committee of the approved applications and notifying the respective beneficiaries of the application-concerning decisions, • Preparing agreements between marszaleks, chairmen of Regional Steering Committee and beneficiaries, • Working in the Public Procurements Committees and, if necessary, accepting responsibility for those procurements, • Supervising the subcontracted work and sending payment orders to the payment agent, • Collecting monitoring and evaluation information in collaboration with local authorities, sending data to the National Steering Committee. • VPIU will serve as RSC secretariats.”148 The Project Appraisal Document presents a draft flow of funds procedure. „Bank will make a preliminary authorized payment to Special Account in the National Bank of Poland and then transfers will be effected against payment instructions from MOF. In the case of an infrastructure development Component, flow of funds takes place from the Special Account to Voivodship Office Finance Departments (expenditure justification shall be sent to Bank within 30 days). If the public finance law is amended, flow of funds shall be in agreement with the respective amendments. When the project packages have been approved by the National Steering Committee and the financing agreements signed by respective Marszaleks (see point 48), Ministry of Finance will transfer financial means in accordance with the official budget procedures and conditions agreed for each Component, as shown in Annex no. 2. VPIU will then prepare cofinancing agreements to be concluded between Marszaleks and gminas. When those agreements have been signed, gminas will start implementation process with the 147 148 Ibidem, p. 6. Operational Manual, p. 5. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 387 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. assisstance of VPIU. In the case of activities connected with building up institutional potential and resource center in the education Component, the responsible ministry will effect payments directly to the service providers. In the Component A (micro-credits), payments will be effected by the National Economy Bank (BGK) through the selected service providers. Ministry of Finance will transfer part of the approved total sum (not more than 20%) as an advance payment and then will be transferring money against fund replenishment applications submitted by VPIU. When 50% of the advance sum have been spent, VPIU will submit the fund replenishment applications to the Ministry of Finance, which will instruct the Bank to replenish the funds from Special Account. Procedures of returning the funds will be established in case savings are achieved or expenditures not qualified for financing. By separating the programming and implementing functions from the payment function, the RDP decision-making process will be subjected to public control and the distribution of funds will be performed in an intelligible and transparent way. This allows to manage the Project more effectively and clarifies the respective scope of responsibility of the local, regional and national decision-making levels.”149 „A monitoring function will be connected with the scope of responsibility of each ministry in the Project. In order to perform that function, each of the ministries will designate one of its departments or an external service provider to carry out the monitoring operations. The teams responsible for monitoring operations will collaborate at the national level with the Project Coordination Unit and will make available the Project realization information in the form of respective indicators. Project progress reports will be submitted by all the lower level units to the Voivodship Project Implementation Units or, in the case of institutional potential building activities, the service provider reports will be submitted to the Project Coordination Unit by design teams in the ministries. The Project Coordination Unit will prepare periodical summary financial reports for the National Steering Committee, Ministry of Finance and Bank. Those reports will be an element of the Management Information System (MIS), which will inform of the goals achieved in relation to the financial means used. The periodical evaluation of the RDP (initially performed every half a year) on the regional and local level has been included in the Project realization schedule to allow continuous Project improvement.”150 The authors of Project management conception underline that „detail administrative procedures, principles of financial management and of payments (…) as the Project realization proceeds (…) may be changed, provided they are not incompatible with this >Operational Manual<”151. Assumptions of the Management Information System (MIS) are presented in the Operational Manual: „In the Project a complex, integrated management information system will be developed, adjusted to the Project needs, which will include financial management information, public orders, training of the administration, project details, 149 Project Appraisal Document, p. 18. Ibidem, p. 18-19. 151 Operational Manual, p. 5. 150 388 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. monitoring and evaluation. The system administrator in the PCU will be responsible for updating and maintenance of the system.”152 8.4 The Poland Flood Emergency Project as a reference to RDP An important aspect of the work on the Project assumptions seems to be the fact that the Project Appraisal Document has been prepared in the World Bank by the same team of experts who prepared the basis for the Poland Flood Emergency Project in 1997. The goal of the Poland Flood Emergency Project was reconstruction of the infrastructure damaged or destroyed by the 1997 flood. The Project had, like the RDP, several Components and Sub-Components. • A – repairs of the municipal and rural infrastructure, • B – construction of anti-flood banks and limiting the flood hazards, including • o B.1. – planning of anti-flood banks, o B.2. – development of a monitoring, forecasting and warning system, o B.4. – prevention and limiting of the flood hazards, C – project management. The project was realized from March 1998 to 31 August 2003. The original completion date – August 2001 – was changed after amendments introduced to the agreement in January 2001. Total value of the credit agreement was USD 200 million and the project tasks were based on the total budget of USD 498.2 million. An important institution for the Project was the World Bank Project Coordination Unit (PCU), responsible for implementation of the accounting and audit system and monitoring the flow of funds during the project. It prepared project task progress reports for the World Bank, based on the information provided by the project realizing parties and by the project beneficiaries. From the Polish government side, members of the Steering Committee were Cabinet Member responsible for repairs of the flood damage and representatives of the Ministry of Finance, Ministry of Environment, Natural Resources and Forestry. Procurements and work orders for the Project were performed in accordance with guidelines given in the World Bank instructions. The Public Procurements law procedures did not apply to the project, as an exception based on an international agreement signed by the government of Poland. Realization of that Project was subjected to analysis and critical evaluation by the Supreme Chamber of Control. These are some of their comments: 152 Operational Manual, p. 7. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 389 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. „The borrower was obliged to make the credit available in a form of subsidies. Such solution was possible for Component A in gminas and powiats, but was not possible, due to the Public Finance law requirements, for Component B in financing the implementation units (Meteorological Institute, Regional Water Management in Krakow, Poznan and Wroclaw).” 153 „The control has shown that realization of the Poland Flood Emergency Project tasks, cofinanced by the World Bank financial means, was unpunctual in a different scale in its individual Components.”154 „Main reason of the delays in realization of the Poland Flood Emergency Project, and its Component B in particular, was insufficient readiness of the implementing units and problems in collaboration with the World Bank.” 155 „The problem observed in the whole controlled period were delays in payments for the tasks co-financed by the World Bank. The reason was too long money transfer time from the borrower to the beneficiary. Average beneficiary’s money transfer awaiting time was 28 days in 1998, 43 days in 1999, 39 days in 2000 and 50 days in 2001, but in extreme cases it reached 224 days. That question was not solved although the World Bank many times demanded the process to take not more than 31 days and the Polish side made efforts to fulfill that demand.” 156 „Collaboration with the World Bank on the credit agreement was not good, particularly in the initial stage. An obstacle were some controversial provisions of the agreement and also, to some extent, not clearly enough defined principles of collaboration between the parties to the agreement. Therefore, the Polish side had difficulties with meeting the Bank requirements. Bank would change the bidding procedures for biddings in progress or would withdraw from an earlier approval of the Component B summary realization schedule. That caused task realization delays. • Neither the credit agreement nor the World Bank instructions regulated the time allowed for individual activities in a procedure and there were cases of long-lasting acceptance of the bidding documents. For instance, the World Bank acceptance procedure of bidding documents for the „Feasibility study of the Racibor reservoir dam” task took 18 months. • On 16 March 1999, the World Bank approved the Component B summary realization schedule and accepted the start of Project realization. But already in June 1999 the World Bank working mission, visiting the Project, submitted reservations to the earlier approved schedules, particularly to work subdivision and bidding procedures. Work on the amendments lasted until July 2000 and the Project realization delay increased by another 12 months.” 157 „The multilevel Project management and supervision system (Steering Committee, the Prime Minister Office, Ministry of Finance, Ministry of Environment, the World Bank Project Coordination Office in Wroclaw, implementation units) partly imposed by the credit agreement provisions, did not guarantee a proper task execution. There was no clear definition of the mutual responsibility, rights and intelligible procedures. Therefore, in the 153 „Information on the results of control of the use of the World Bank credit 4264-POL for “Repair of the July 1997 flood damage” project” Wroclaw, December 2002, p. 28. 154 Ibidem, p. 7. 155 Ibidem, p. 6. 156 Ibidem, p. 9-10. 157 Ibidem, p. 10. 390 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. opinion of NIK, the financial management and supervision of the Poland Flood Emergency Project were inadequate, which is confirmed by the observed improper management of the credit and associated budgetary means and also by the delays in the fund flow. The supervision was particularly inadequate in the realization of Component B of the Project.” 158 „The best level of the Project Component B supervision was provided by the Steering Committee chaired by the Minister responsible for repairs of the flood damage, who became more active, however, only at the beginning of 1999. Inadequately worked the World Bank Project Coordination Office in Wroclaw, established for coordination and supervising the credit means spending and also for realizing the credit agreement provisions. That unit did not have a legal status in the Polish legal system and in the Prime Minister Office statutes. It functioned for 3 years without an approved organizational constitution (that was a responsibility of the alternating Office directors), the organizational structure, employment and remuneration methods were different than those described in the credit agreement and in the years 2000-2001 the Office had no financial plans prepared or approved.”159 „Also questionable is the way the supervision and control were executed by the Ministry of Environment, which directly supervised realization of Component B >construction of the flood banks and limiting flood hazard<. No control system was created for the tasks financed from the World Bank credit and performing the supervisor functions by just one person, as an additional duty, did not guarantee a proper monitoring of task realization”.160 Summing up – some important aspects of the Poland Flood Emergency Project were similar to the Rural Development Project: • It had a multi-Component structure, • Several ministries were involved in it on the Polish government side, • Special institutions were set up to perform effectively the key administrative tasks, • The World Bank goods and services procurement bidding procedures dominated over the Polish legal procedures, • Payment procedures were subjected to the requirements of the Public Finance law. A series of objections were formulated against the Project, regarding the effectiveness of management and of task execution. The Rural Development Project designers proposed an organizational formula that to a significant extent was a continuation of the one used in FEP. They also proposed distinct improvements on it so as to avoid recurrence of mistakes made during the FEP implementation. The evaluators will return to this question further in this chapter. 158 Ibidem, p. 10-11. Ibidem, p. 11. 160 Ibidem, p. 11. 159 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 391 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. 8.5 Selected problems management 8.5.1 of the Rural Development Project Taking decisions on the changes to the RDP principles RDP was designed as a multi-sector project and its pioneering character was underlined right from the beginning. Therefore, some flexibility of the RDP procedures was adopted and it was also assumed that the Project would evolve during realization and the changes would be made along the pre-designed paths. The basic RDP document was the Operational Manual. Changes to that document had to be approved by the NSC. Changes to the Operational Manual couId be initiated by the RDP -involved institutions and, indirectly, by the beneficiaries. World Bank Figure 83. Diagram of making decisions on changes to Operational Manual Initiators of changes PCU National Steering Commitee Initially opinions the proposal "No objection" Approves changes to Operational Manual Opinion the proposal, consults it with involved institutions, passes to WB to opinion Prepares the proposal of change & submits it to NSC Includes changes in the Operational Manual, distributes knowledge about changes Propose changes to Operational Manual Source: Own study of RDP documents Changes to the Operational Manual, introduced by the NSC during the RDP, concerned: 392 • No data are available for 2000 • NSC resolution of 27 June 2002, Sub-Component B1 – adding annex 3.2; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. • NSC resolution of 16 January 2003, Sub-Component B1 – adding annex 3.3; • NSC resolutions of 5 April 2004, finance management system and realization of SubComponent B1 (annex 3.2); • NSC resolution of 26 April 2004, amendment to annex 3.3. 8.5.2 Decision-making process of granting subsidies to beneficiaries There are 3 main groups of beneficiaries in the RDP: 1. Individual persons – in the case of Components A and B1 – requalifying / reorientation program and B2 – Education Resources Center. 2. Voivodships, gminas, powiats, Voivodship and Powiat Labor Offices for economic and labor market research in Sub-Component B1. 3. Gminas and powiats (local communities) – in Components B2, B3 and C. In the case of individual persons, there were two separate decision making procedures for granting the subsidies. In Component A a subsidy for beneficiary had two forms:: • loan granted by loanable funds, co-financed from the RDP means; • non-repayable subsidy financed from the RDP means. Subsidies were granted by foundations (loanable funds) selected by bidding. They had their own structures in voivodships. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 393 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Diagram of making decisions on granting subsidies to the Component A beneficiaries Loan comission Figure 84. Inspector in RDF/FDPA Decides about granting of microloan or subsidy Registers Rejestruje the application wniosek Loan officer or inspector confirm the signature Beneficiary fills the application form consulting it with loan officer or inspector Opinions the Opiniuje wniosek application Sign the contract Beneficiary or loan groups Loan officer Opinions the application Source: Own study of RDP documents In the Sub-Component B1 the subsidy had an indirect character – services provided by local firms were financed fully (most often) or partly from the RDP funds. The services were rendered in the following spheres: • vocational training; • employment opportunities (vocational guidance); • temporary employment (refunding part of the employment costs to the employer); • entrepreneurship supporting centers (help in setting up and running a business unit); • entrepreneurship incubators; • local development planning. Then the decision making procedure for granting subsidies, i.e. providing free service, was made in two stages: 394 • at the voivodship level (for greater orders also at national level: Ministry of Economy and Labor, World Bank) – decision on realization of a specific service and selection of the service provider; • at the beneficiary level – decision on qualifying a beneficiary for free services in the Project. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. At the voivodship level, the decision process was based on the competition rules in accordance with the project guidelines given in annex 3.3 to the Operational Manual. Diagram of making decisions on concluding agreements with service providers in Sub-Component B1 Service providers Figure 85. Negotiate contracts In case of contract value > 35 tys. US$ i <= 70 tys. US$ approves Prepare subcomponent's promotion & monitoring system In case of contract value > 70 tys. US$ approves Analyse the needs & approve the demand RSC Local governments Organise contests in order to find service providers Opinions on the contracts or sends to PT MoEL or WB WB Program Team of Ministry of Economy & Labor Voivodship consultant VLO Gather the needs and pass it to RSC Signing of contracts Identify the needs of local communities Source: Own study of RDP documents Decision on qualifying a beneficiary for a service was taken by the Voivodship Labor Office. The only selection criterion was compatibility of the service applicants with the Operational Manual together with annex 3.3 and with the criteria adopted by the local RSC. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 395 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. The Internet Educational Resources Center – Scholaris site, generally accessible free service – each interested teacher, student, pupil may use the site. Selection of the service provider was made in a QCBS bidding organized by the Ministry of Economy and Labor. The main receiver of the economy and labor market research in Sub-Component B1 was the Program Team of the Ministry of Economy and Labor (formerly Ministry of Labor and Social Policy). Each of the interested beneficiaries (voivodship offices, powiats, gminas, Voivodship and Powiat Labor Offices) could use the effect of work – voivodship reports. Selection of the service provider was made in a bidding organized by Ministry of Economy and Labor according to the QCBS procedure. The research performer was a consortium of companies – Polish Network of Business Consultants (PSDB S.A.), Market Economy Research Institute (IBnGR), IPSOS – Demoskop. The market research reports were sent by PSDB for acceptance by National Steering Committee. Besides, the results were presented at conferences organized for the Sub-Component B1 implementing institutions and in an Economy and Labor Market forum. Beneficiaries of the Components B2, B3 and C were local communities of gminas and powiats (Component C). The decision making procedures in those Components were different, according to the place of taking decisions on the selection of beneficiaries or place of selecting the service provider. Three basic decision making models may be distinguished: • in Sub-Component B2 training and procurement, B3 training program – decisions at the voivodship level based on the gmina applications and on central biddings; • in Sub-Component B3 IDP and dissemination program – decisions at the national level based on the powiat/gmina participation in IDP, dissemination program for all gminas in Poland based on central biddings; • in Sub-Component B2 school repairs and modernization and in Component C – decisions at the voivodship level based on gmina and powiat applications and on local gmina level biddings. Table 125. Decision-making models in qualifying a beneficiary in Components B2, B3 and C Activity in Component / SubComponent B2 training and procurements B3 Training Program B3 IDP, B3 IDP Dissemination Program B2 repairs and modernizations, C infrastructural investments Decision on qualifying a beneficiary Bidding organizer RSC/VPIU by demand / gmina applications Program Teams in MoNES and MIAPA MIAPA Program Team by local government unit (LGU) applications MIAPA Program Team RSC / VPIU by LGU applications Gminas / gmina associations, powiats Source: Own study of RDP documents Beneficiaries of procurement and training in Sub-Component B2 were gminas and their communities, in particular rural school pupils and their families, headmasters and teachers of those schools. Decision on the selection of beneficiaries – schools, teachers, headmasters and other education sector workers at the gmina level was usually taken by 396 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. school headmasters (submission of needs to gmina). Gminas collected those applications and completed a final list of needs, submitted then to the VPIU. Regional Steering Committees played key role in taking decisions on selecting gmina applications for realization. They completed and approved ranking lists based on earlier established criteria and the formal appraisal reports issued by the VPIUs. Diagram of making decisions on realization of educational and training tasks in Sub-Component B2 Fill in trainingdidactic applications Basing on preset criteria & VPIU reports create ranking lists VPIU Evaluate offers & choose the best ones, negotiate Prepare bidding documentation On behalf of VPIU invites providers to apply offers Apply offers Voivodship Administration Office PCU Sends to voivodships a list of programmes, didactic aids, technical specifications & ToR Evaluate applications & present conclusions & evaluation reports to RSC Providers Invite gminas to apply for grants Program Team in Ministry of National Education & Sports RSC Gminas Figure 86. Sign the contracts Source: Own study of RDP documents In Sub-Component B3, an IDP pilot program was planned for gminas in 3 powiats in Poland. Qualifying of powiats/gminas for the IDP was decided by the MIAPA Program Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 397 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Team supported by consultants. Selection was based on questionnaires sent by the interested powiats. After verification of the questionnaires, a short list of 8 powiats was set up, of which 3 were selected to implement the IDP. An IDP consultant was chosen in accordance with the QCBS procedure organized by the MIAPA Program Team. The IDP was realized by a consortium of Canadian Urban Institute and the Malopolska Public Administration School. In B3 Training Programs, the following teams (composed of persons responsible for management and decision making in the local government units) could submit applications for training participation: • from rural gminas; • from municipal-rural gminas; • from municipal gminas with population of up to 15,000; • from municipal gminas with 15,000 – 40,000 inhabitants, on the condition that they would qualify only when there were free places left after recruitment of other entitled local government units; • from the Marszalek Office representing teams. RSC took decisions on qualifying teams to the training program and drew up lists of applications, based on evaluations and amount of credit allocated by VPIU. The local government unit qualification process was carried out in collaboration with the training provider – the Foundation for Local Democracy Development (FRDL). FRDL was chosen in a QCBS bidding organized by the MIAPA Program Team. The IDP result dissemination program in Sub-Component B3 included all gminas in Poland. The program was realized by MIAPA supported by four local government associations – Association of Polish Cities, Association of Rural Gminas, Association of Powiats and Union of Polish Towns. A similar procedure of selecting the beneficiaries was used in Sub-Component B2 repairs and modernizations and in Component C. In accordance with that procedure, the local government units submitted investment applications to VPIU. VPIU (in Component C together with ARMA) prepared a preliminary evaluation of / opinion on those applications in accordance with criteria adopted by the RSC. The final ranking lists and lists of accepted applications were adopted by the respective voivodship RSC. The local government units organized biddings according to the scale and character of work, as required by the RDP Public Procurement Book. As majority of the investments were relatively small scale undertakings, the biddings were of the MW type (Components C and B2) and LVMW type (Sub-Component B2 only). 398 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Diagram of making decisions on investment realization in SubComponent B2 and Component C Decide about the evaluation criteria Inform local governments and invite to the PRDP Basing on VPIU/ARMA opinions decide about ranking list of projects to be financed Opinion the applications basing on the criteria decided by RCS Prepare & submit investment applications to VPIU (B2) or ARMA (C) Contractor Local governments ARMA (component C only) VPIU Regional Steering Commitee Figure 87. Conduct biddings according to WB rules Evaluate offers, choose the contractors Local governments sign contracts with the contractors Submit offers Source: Own study of RDP documents 8.5.3 Finance management in RDP The finance management process in RDP was defined in Annex 2 to the Operational Manual of 11 April 2000 - „Finance Management System”. The document describes tasks of the individual Project institutions and the respective financing procedures. Procedures contained in annex 2 to the Operational Manual of 11 April 2000 evolved considerably during the RDP. The evolution was a result of overcoming, by the local government units and other RDP implementing parties, the arising problems and inaccuracies in the fund flows. The essential path of changes was modification of the procedures in order to prevent payment delays and also simplification of the annex provisions to make them easier to apply and construe. More or less at the turn of 2002 the procedures took shape that remained unchanged until the end of the Project. All the changes in the financing procedure until April 2004 did not have any formal effect in the change of Operational Manual. The RDP implementing units had a knowledge of procedures in force, but there was no single, unequivocal and legal written source of that information. On 5 April 2004 the National Steering Committee amended Annex 2 to the Operational Manual of 11 April 2000. In fact, the new version of Annex 2 was a Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 399 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. completely new document – in the opinion of evaluators more precise and clear than the old version – describing the RDP financing procedures. The RDP fund flow process had to be in agreement with provisions of the Public Finance law of 26 November 1998 with later amendments. In view of that, the World Bank did not have the right to transfer financial means directly to beneficiaries. Those financial means had to pass through the special account of the Ministry of Finance and then according to the Public Finance law general principles of distribution of budgetary means. The adopted procedure was based on the Polish Public Finance law and on the World Bank procedures. In developing the fund flow procedures the World Bank experience gained in the Poland Flood Emergency Project was used. Procedures for that Project were written in the Project Appraisal Document of 1 December 1997. Proposals in the RDP PAD were similar to those in the flood project PAD. The Flood Emergency Project had two main objectives: • Reconstruction of the essential infrastructure in the inundation area by repairing the structures and objects important from the economic and social point of view; • Preventing the risk and negative effects of flood in the future. PFEP consisted of two „material” Components: that dealing with the gmina infrastructural investments and that of waterway and anti-flood infrastructure investments managed by the Regional Water Management Boards. With great simplification, the whole project was similar to Component C in RDP, many investments involved relatively small, by the World Bank standards, sums of money. The project was more uniform than RDP, its activities were concentrated in the Ministry of Flood Damage Reconstruction and also Ministry of Agriculture and Ministry of Environment. The project was managed by Project Coordination Unit in the Ministry of Flood Damage Reconstruction and its tasks were similar to those of the PCU in RDP. The basic legacy taken over by RDP from PFEP were the bidding procedures, based on the World Bank Instruction on the public procurements financed from the IBRD loans and IDA credits. In PFEP, in order to adjust the WB standard bidding solutions, used mainly in large international biddings, to the Polish conditions, those procedures were supplemented with the National Shopping (NS), National Competitive Bidding (NCB) and Minor Work (MW) type of bidding. In our opinion, effective execution of most of the biddings in PFEP was only possible after introducing such solutions. In RDP it was decided to use the World Bank suggested bidding procedures, taken over entirely from PFEP. In the interviews, during RDP evaluation, with the Project implementing institutions, drawbacks of those solutions were more often indicated than the advantages. The drawbacks mentioned were the following:: 400 • the necessity of learning and implementing by gminas, powiats and RDP institutions of complicated procedures, completely different from the Polish Public Procurement law; • that acquired bidding knowledge will be of no use in the future; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. • some interviewers indicated unclear mutual relation of the World Bank bidding procedures and the Polish legislation; • too complicated bidding procedures for small value orders and often occurring problems with solving the biddings – this could be seen in the work procurements in Sub-Component B2 and was partly solved by introducing the Little Value Minor Work (LVMW) procedure; • necessity of obtaining the World Bank „no objection” decision in key stages of some large value bidding procedures, which considerably prolonged the bidding process. As an advantage of the World Bank solutions, the rigidity of procedures was mentioned and availability of ready forms for bidding documents, which (in the opinion of some respondents) simplified the bidding process. Most of the interviewed persons were of the opinion that the RDP objectives could have been much easier achieved with the use of the Polish Public Procurement procedures, particularly after amendments adjusting the Polish law to the European Union legislation. But the question whether „pure” World Bank procedures or the changed PFEP procedures would have been better for RDP has an obvious answer. Without the use of PFEP procedures, realization of RDP, at least in the Components C infrastructure and B2 repairs and modernization, would have been extremely difficult. In the RDP, the money payments were effected through budget procedures, assuming participation in the financing process of the national and voivodship government structures – Ministry of Finance and Voivodship Office Finance Departments. The following RDP finance management procedures were in use: 1. Replenishment of the special account means and transfer of credit money to the state budget. 2. Payment applications. 3. Payments. 4. Accounting. Procedure 1 functioned at the Project national level and procedures 1, 2, 3 were appropriate for Components A, B (B1, B2, B3), C and D. Within points 2, 3 and 4 there were 11 modes of money flow: • • Component A: o micro-loans and service provider remunerations; o single subsidies; o Component A management; Sub-Component B1: o reorientation / requalifying program; Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 401 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. o • • Sub-Component B2: o school repairs, school network consolidation and training – educational tasks; o technical assistance and equipment for the Internet Center, technical assistance in implementation and monitoring; Sub-Component B3: o • • labor market research and Sub-Component B1 management; pilot IDP, training program, technical assistance and deliveries of equipment for the IDP units, dissemination program; Component C: o investment tasks; o training and technical assistance; Component D: o Project management. All the above mentioned modes of money flow had many elements in common, they differed by specific solutions for the scope and character of the individual tasks within Components. Detailed descriptions of the procedures are given in Annex 2 to the Operational Manual of 9 March 2000. 8.5.4 Replenishment of the special account means and transfer of credit money to the state budget Key role in this procedure was played by the Ministry of Finance, which applied on behalf of the government to the World Bank for replenishment of the special account means and then it had the means at its disposal. 402 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Figure 88. Replenishment of the special account means and transfer of credit money to the state budget PCU on behalf of MARD Ministry of Finance administrator of the Special Account in NBP) World Bank Approval of the application & replenishment of the special account means Approval for the application by Minister of Finance Application to the WB for replenishment of the special account means MF's order to replenish national budget account from the special account Preparation of application with expenditure account, application for acceptance of the Minister of ARD Source: Own study of the Annex 2 to the Operational Manual and other RDP documents 8.5.5 Applying for payment General procedure of applying for payment, used in all Components except A – microloans. The procedure is presented in 3 variants: • when the executor was selected in a bidding at the ministerial level; • when the executor was selected at gmina/powiat level; • in Sub-Component B1, where executor sent invoices to the Voivodship Labor Office. Additionally, in Component D, PCU applied to MF, with prior acceptance by MARD. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 403 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Applying for payment in RDP – general procedure Decision on payment Prepares aggregated application Prepares aggregated application Prepares application Ministerstwo Gmina lub powiat odpowiedzialne za (beneficjent) dany przetarg VLO VPIU on behalf of Marszalek PCU on behalf of MoF (administrator of the MARD special account in NBP) Figure 89. Prepares application Sends invoice to PCU/ prepares application central bidding Contractor local bidding Invoice subcomponent B1 Source: Own study of the Annex 2 to the Operational Manual and other RDP documents Specific feature of applying for payment in Component A – micro-loans is that there is no invoicing in the process. The loanable funds (service providers) collect applications from beneficiaries and submit a summary application to BGK. BGK prepares applications to Ministry of Economy and Labor (MEL), then the procedure is the same as the general procedure. The loanable fund financing is in agreement with the general procedure of applying for payment. 404 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. 8.5.6 Payments The payment process in RDP was the reverse path of the applying for payment process, with all the intermediate levels except PCU which had no own account. The diagram in the Figure below may be used for most activities and Components. The procedure is presented in 3 variants: • when the executor was selected in a bidding at the ministerial level; • when the executor was selected at gmina/powiat level; • in Sub-Component B1, where executor sent invoices to the Voivodship Labor Office. Payments in RDP – general procedure Decision about payment Transfer of means to the accounts of Ministries/Voivo dship Governments Transfer of means to Marszaleks Offices accounts Transfer of means to gminas/powiats or VLO biddings in gminas & powiats subcomponent B1 VLO Transfer of means to the contractor's accounts Gmina or powiat (beneficiary) Ministry responsible for bidding local biddings central biddings Marszaleks Offices Marszaleks Offices Ministry of Finance Finance (administrator of the Departments special account in NBP) Figure 90. Transfer of means to the contractor's accounts Transfer of means to the contractor's accounts Source: Own study of the Annex 2 to the Operational Manual and other RDP documents As in the applying for payment, Component A is specific. Depending on the product (micro-loan and loanable fund remunerations or a single subsidy), the procedure was different. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 405 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Payments in Component A Payment decision Transfer of means to accounts of MoEL/FD VAO Microloans & financing of FDPA & RDF Subsidies Transfer of means to FD VAO accounts Transfer of means to Marszaleks Offices accounts Transfer of means on FDPA & RDF accounts Account of MoEL in BGK Marszaleks Offices Ministry MoF (administrator of the Financial Departments responsible for the special account in NBP) in VAO bidding Figure 91. FDPA, RDF Transfer of means on FDPA & RDF accounts Payments to beneficiaries Source: Own study of the Annex 2 to the Operational Manual and other RDP documents 8.5.7 Accounting of payments The payment accounting procedure in RDP consisted in entering the transfer information in the MIS database by the institutions responsible for payments to beneficiaries (Loanable Funds and BGK/MEL in Component A, VLO and VPIU in B1, VPIU in local biddings and ministries in the national level biddings) and transferring that database to PCU. PCU combined the information from various databases into the central MIS database. The frequency of sending the MIS database to PCU, i.e. length of the accounting period, changed during the RDP – from about one month at the beginning of RDP to 2 weeks (after the NSC resolution of 30 April 2002). At the RDP initial stage, the adopted accounting system was one of the reasons of delays in the financing process, due to the sluggishness of gminas in accounting with VPIU for 406 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. investment expenditures and also due to the prolonged time of decision making by the MF. Therefore, remedial measures were taken to restore the RDP financing fluidity. One of the PCU roles in RDP was participation in the financial planning. The financial planning procedure was based on regulations and procedures of the Polish Budget Planning law and on the following stages: 1. RDP had annual financial plans, in accordance with the budget cycle. Implementation of the financial plan consisted in assigning means for individual voivodships by NSC, based on applications submitted to PCU. 2. PCU updated the performance data in MIS (monthly cycle). 3. If a voivodship plan fulfillment was lower than 60%, PCU asked the Voivodship Marszalek to release the money. 4. If Marszalek released money, PCU asked MARD to correct the financial plan. 5. MARD, authorized by PCU, corrected the financial plan. 6. If the changes might have an impact on the credit agreement limits then acceptance was necessary of the other party – the World Bank. 8.5.8 Effecting payments in RDP It may be seen from the MIS database information on the Project payments how the individual Components were realized. It is also possible to obtain an overall picture of RDP activities in time. Figure 92. RDP monthly payments in thous. of PLN 80 000 70 000 60 000 50 000 40 000 30 000 20 000 10 000 0 01 2000 07 2000 01 2001 07 2001 01 2002 07 2002 01 2003 07 2003 01 2004 07 2004 01 2005 Source: MIS database received from PCU The above diagram shows that the actual RDP realization phase started in the second half of 2001, when the Component C investments started. The annual money spending cycIe is clearly seen, correlated with the annual budget cycle of the ministries (including Ministry of Finance) and local government units. In that cycle there is a financial “pit” at Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 407 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. the beginning of each year and majority of the money means are spent in the second half of the year. That “pit”, in the opinion of some respondents, was an effect, at least partly, of a limited decision making capability of the Ministry of Finance as regards money spending at the turn of the calendar years. Also clearly seen is the accelerated process of money spending in the second half of 2004 as the RDP approached the end. Figure 93. Accrued payments in RDP, by Components, in millions of PLN 600 500 400 Component C Subcomponent B2 300 200 100 Subkomponent B1 Component A Subcomponent B3 Component D Fe br ua ry -0 0 M ay -0 Au 0 gu st N -0 ov 0 em be r- 0 Fe 0 br ua ry -0 1 M ay -0 Au 1 gu st N -0 ov 1 em be r- 0 Fe 1 br ua ry -0 2 M ay -0 Au 2 gu st N -0 ov 2 em be r Fe -0 2 br ua ry -0 3 M ay -0 Au 3 gu st N ov 03 em be r Fe -0 3 br ua ry -0 4 M ay -0 Au 4 gu st N -0 ov 4 em be r- 0 Fe 4 br ua ry -0 5 0 Source: MIS database received from PCU The diagram shows that the greatest part of the total sum of payments went to Component C. Also a substantial part was taken by Sub-Component B2, where very important was assigning the Project reserve fund means to that Sub-Component. It is also interesting to note the sequence of the individual Components in time – Component C was realized as the first – initial significant payments were effected at the end of 2001. That Component was also finished as the first one. The diagram suggests that Component C was in a sense separate from other RDP Components, well ahead in time. 408 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Figure 94. Accrued payments in RDP, in Component A and Sub-Component B1, in millions of PLN 160 140 120 Subcomponent B1 100 80 60 40 20 Component A Fe br ua ry -0 0 Au gu st -0 Fe 0 br ua ry -0 1 Au gu st -0 Fe 1 br ua ry -0 2 Au gu st -0 Fe 2 br ua ry -0 3 Au gu st -0 Fe 3 br ua ry -0 4 Au gu st -0 Fe 4 br ua ry -0 5 0 Source: MIS database received from PCU Comparing Component A and Sub-Component B1, which were in the hands of the MEL Program Team, one can see that they were not performed in parallel. Realization of SubComponent B1 was started in the first half of 2002 and Component A (micro-loans and subsidies) was fully operational a year later. But Sub-Component B1, from the point of view of payments, ended later than Component A – at the turn of 2004. Most micro-loans were paid by the end of the first half of 2004. One may agree with the opinion that if the two Components had been better coordinated in time, much better synergy effect would have been achieved. Presumably those applying for micro-loans would have a chance of participating in various forms of training available in Sub-Component B1. However, we had signals that target groups of the two Components were different – persons in Component A performed business activity within their qualifications and participants of Sub-Component B1 received training in new skills to find a job. Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. 409 Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Figure 95. Accrued payments in RDP, in Component C and Sub-Component B3, in millions of PLN 600 500 Component C 400 300 200 Subcomponent B3 100 Fe br ua r y00 Au gu st -0 Fe 0 br ua ry -0 1 Au gu st -0 Fe 1 br ua ry -0 2 Au gu st -0 Fe 2 br ua ry -0 3 Au gu st -0 Fe 3 br ua ry -0 4 Au gu st -0 Fe 4 br ua ry -0 5 0 Source: MIS database received from PCU Comparing the realization time of Component C – infrastructural investments and SubComponent B3 – development of institutional potential, which prepared the local government units for making use of extra-budgetary investment financing sources, one can notice that those two Components were carried out in a reversed, illogical sequence. In our opinion, realizing the Sub-Component B3 first would be better justified and would prepare a better realization of Component C. We can imagine that potentially more gminas could apply for money, applications would be better prepared, without delays and with fewer errors, and the whole Component C would be more effective. There were preparatory instructions for gminas in Component C before applications but they were narrow in scope comparing with the complex approach in Sub-Component B3. It has to be remembered, though, that the RDP Component C has ended with a success. 410 Consortium: DORADCA Consultants Ltd. Sp. z o.o., Instytut Spraw Publicznych, PBS Sp. z o.o., Proeko Ltd. Final Evaluation Study of the Rural Development Project – Evaluation of Project management Tomasz Majewski, Maciej Gajewski, Ewa Gąsiorowska, Piotr Wróbel, Ph. D. Figure 96. Monthly payments in RDP, in Component A, in thousands of PLN 12 000 10 000 8 000 6 000 4 000 2 000 0 2001 2002 2003 2004 2005 Source: MIS database received from PCU In Component A, realization of the basic tasks, and therefore greater expenditures, began as the last of all RDP Components. That was caused by difficulties with selecting the service providers – loanable funds. Those problems were of a procedural character and were connected with the World Bank QCBS procedure. Its provisions were not adjusted to the activities in that Component. When Component A started (turn of 2002), relatively high rate of money spending could be observed. Figure 97. Monthly payments in RDP, in Sub-Component B1, in thousands of PLN 10 000 9 000 8 000 7 000 6 000 5 000 4 000 3 000 2 000 1 000 0 2001 2002 2003 2004 2005 Source: MIS database received from PCU Expenditures in Sub-Component B1 started at the end of 2001 but at the beginning of 2002 a clear financial “low spot” can be