Independent Evaluation Group Studies
8 total works
The global economy is passing through a period of relative calm, but the situation remains fragile in many areas. Developed economies are gradually strengthening, but growth in developing economies has slowed. Successful implementation of World Bank Group operations in this demanding environment will require more effective & efficiency with limited resources.
How well has the World Bank generated, accessed, and used learning and knowledge in its lending operations, and what is the scope for improving how it does so? As the world s leading development finance agency, the World Bank has, in principle, an unrivaled opportunity to promote learning and knowledge sharing about development effectiveness. Bank lending has fallen in relation to developing country gross domestic product. To remain relevant, the Bank must improve the quality of its services; learning and knowledge offer an important competitive edge. The challenge is to become better at learning from lending, responding more quickly to lessons from experience, and being more alert to the creation and use of cutting-edge knowledge.
This first of two Independent Evaluation Group (IEG) evaluations on this topic, pulls back the curtain to reveal the perspective from within the World Bank, with findings drawn from a wide range of sources: the academic and management literature on organizational learning; Bank strategies and studies; Bank employee surveys; an IEG survey of Bank staff; interviews and focus groups, including consultations with other development agencies; IEG reviews of project documents as well as project evaluations and studies; and aggregate data on the learning section of the annual Overall Performance Evaluation of Bank staff. It shows that although, in general terms, the staff perceive the Bank to be committed to learning and knowledge sharing as attested by surveys with respect to the particular case of learning in lending, the culture and systems of the Bank, the incentives it offers employees, and the signals from managers are not as effective as they could be. The Bank s organizational structure has been revamped several times, notably in 1987 and 1996, when the shift to the matrix organization took place.
These changes have not led to a significant change in learning in lending because they touched neither the culture nor the incentives. It remains to be seen if the latest structural change the introduction of Global Practices will avoid past pitfalls and be more effective in creating the right culture and incentives for learning and knowledge sharing
This first of two Independent Evaluation Group (IEG) evaluations on this topic, pulls back the curtain to reveal the perspective from within the World Bank, with findings drawn from a wide range of sources: the academic and management literature on organizational learning; Bank strategies and studies; Bank employee surveys; an IEG survey of Bank staff; interviews and focus groups, including consultations with other development agencies; IEG reviews of project documents as well as project evaluations and studies; and aggregate data on the learning section of the annual Overall Performance Evaluation of Bank staff. It shows that although, in general terms, the staff perceive the Bank to be committed to learning and knowledge sharing as attested by surveys with respect to the particular case of learning in lending, the culture and systems of the Bank, the incentives it offers employees, and the signals from managers are not as effective as they could be. The Bank s organizational structure has been revamped several times, notably in 1987 and 1996, when the shift to the matrix organization took place.
These changes have not led to a significant change in learning in lending because they touched neither the culture nor the incentives. It remains to be seen if the latest structural change the introduction of Global Practices will avoid past pitfalls and be more effective in creating the right culture and incentives for learning and knowledge sharing
This evaluation examines the relevance and effectiveness of Poverty Reduction Support Credits (PRSCs), introduced by the Bank in early 2001 to support comprehensive growth, improve social conditions, and reduce poverty in IDA countries. PRSCs were intended to allow greater country-ownership, provide more predictable annual support, exhibit more flexible conditionality, and strengthen budget processes in a results-based framework. By September 2009, the Bank had approved 99 PRSCs totaling some $7.5 billion and representing 38% percent of IDA policy based lending. The evaluation finds that in terms of process, PRSCs were effective in easing conditionality, increasing country ownership and aid predictability, stimulating dialogue between central and sectoral ministries, and improving donor harmonization. In terms of content, PRSCs succeeded in emphasizing public sector management and pro-poor service delivery. Yet in terms of results, it is difficult to distinguish growth and poverty outcomes in countries with PRSCs from other better performing IDA countries.
There is scope for further simplifying the language of conditionality and underpinning PRSCs with better pro-poor growth diagnostics. PRSCs can also strengthen their results frameworks and limit sector policy content in multi-sector DPLs to high-level or cross-cutting issues. Today, Bank policy has subsumed PRSCs under the broader mantle of Development Policy Lending and the rationale for a separate 'brand name' although differences linger from the past. Since PRSCs and other policy-based lending have gradually converged in design, remaining differences compared to other Development Policy Loans should be clearly spelled out, or the separate PRSC brand name should be phased out.
There is scope for further simplifying the language of conditionality and underpinning PRSCs with better pro-poor growth diagnostics. PRSCs can also strengthen their results frameworks and limit sector policy content in multi-sector DPLs to high-level or cross-cutting issues. Today, Bank policy has subsumed PRSCs under the broader mantle of Development Policy Lending and the rationale for a separate 'brand name' although differences linger from the past. Since PRSCs and other policy-based lending have gradually converged in design, remaining differences compared to other Development Policy Loans should be clearly spelled out, or the separate PRSC brand name should be phased out.
Analyzing the Effects of Policy Reform on the Poor
by Soniya Carvalho and World Bank
Published 5 May 2010
This IEG evaluation, requested by the World Bank's Board of Executive Directors, represents the first independent evaluation of the PSIA experience. The evaluation finds that: - The PSIA approach has appropriately emphasized the importance of assessing the distributional impact of policy actions, understanding institutional and political constraints to development, and building domestic ownership for reforms - PSIAs have not always explicitly stated their operational objectives (i.e., informing country policies, informing Bank operations, and/or contributing to country capacity) - PSIAs have had limited ownership by Bank staff and managers and have often not been effectively integrated into country assistance programs - Quality assurance and Monitoring and Evaluation of the overall effectiveness of PSIAs have been weak The evaluation recommends that the World Bank: - Ensure that Bank staff understand what the PSIA approach is and when to use it - Clarify the operational objectives of each PSIA and tailor the approach and timeline to those objectives - Improve integration of the PSIA into the Bank's country assistance program by requiring that all earmarked funding for PSIAs be matched by a substantial contribution from the country unit budgets - Strengthen PSIA effectiveness through enhanced quality assurance
The development paradigm has shifted toward private investment, and the private sector has become central in development strategies. There is much to be learned about how to effectively facilitate and mobilise private sector contributions to development. Effective monitoring and evaluation (M and E) systems are critical for learning to catalyse private sector development. In line with this advance, the International Finance Corporation and the Multilateral Investment Guarantee Agency are developing and refining their M and E efforts.
In this Biennial Report on Operations Evaluation, the Independent Evaluation Group takes stock of the evolution of the M and E systems in the two organisations, assessing their adequacy, coverage, and quality, as well as their respective results measurement systems. IEG acknowledges progress by the two institutions. IFC has advanced its systems for gathering, analysing, and applying project information and has strengthened its coverage of indicators that measure results. Information from M and E has become more prominent in its business decisions. However, the institution’s corporate goals are built on indicators of client reach that cannot be solely attributable to IFC, so there is no credible articulation of IFC’s impact. MIGA has introduced self-evaluation of its projects and started gathering some standard development indicators. As a result, individual learning is taking place in the institution.
The report shows the importance of IFC and MIGA managements continuing their efforts to deepen M and E and improve their systems. To gain the full benefit of learning from evidence that M and E brings to light, key areas need improvement. IEG offers recommendations for IFC regarding quality, verification of data, and tracing effects. For MIGA, IEG notes that it needs to adapt and streamline its evaluation approach to fit its business practices.
In this Biennial Report on Operations Evaluation, the Independent Evaluation Group takes stock of the evolution of the M and E systems in the two organisations, assessing their adequacy, coverage, and quality, as well as their respective results measurement systems. IEG acknowledges progress by the two institutions. IFC has advanced its systems for gathering, analysing, and applying project information and has strengthened its coverage of indicators that measure results. Information from M and E has become more prominent in its business decisions. However, the institution’s corporate goals are built on indicators of client reach that cannot be solely attributable to IFC, so there is no credible articulation of IFC’s impact. MIGA has introduced self-evaluation of its projects and started gathering some standard development indicators. As a result, individual learning is taking place in the institution.
The report shows the importance of IFC and MIGA managements continuing their efforts to deepen M and E and improve their systems. To gain the full benefit of learning from evidence that M and E brings to light, key areas need improvement. IEG offers recommendations for IFC regarding quality, verification of data, and tracing effects. For MIGA, IEG notes that it needs to adapt and streamline its evaluation approach to fit its business practices.
Evaluation of the International Finance Corporation's Global Trade Finance Program, 2006-12
by World Bank
Published 30 September 2013
As part of its strategy to support global trade, the World Bank Group seeks to enhance trade finance in emerging markets. In 2005 the International Finance Corporation (IFC), part of the Bank Group, introduced the Global Trade Finance Program (GTFP) to support the extension of trade finance to underserved clients globally. This IEG evaluation found that overall, the GTFP was a relevant response to the demand to reduce risk in trade finance in emerging markets. The programme significantly improved IFC’s engagement in trade finance by introducing an open network of banks and a quick, flexible response platform to support the supply of trade finance.
IEG’s evaluation covers the programme’s operations from its inception in 2005 through FY2012. The programme grew from a $500 million annual commitment to $5 billion in FY12. It accounted for 39 percent of total IFC commitments and has low costs—it accounted for 2.4 percent of IFC’s capital use and 1.2 percent of its staff costs and has had no claims to date. It is profitable as well, although not to the extent originally expected, accounting for 0.6 percent of IFC’s net profit. IEG found that the GTFP has particular additionality among higher-risk countries. In its early years, it was concentrated in these countries, particularly in Africa. During the global crisis, the programme risk-mitigation instrument became relevant in much broader markets. Client feedback on the programme has been positive.
In its evaluation IEG does offer several recommendations to enhance its effectiveness, including on issues of transparency and reporting methods, as well as expanding the share of the programme in needier markets. For development professionals, the lessons in this evaluation can be applied to private sector development situations, particularly mitigation of financing risks in emerging markets.
IEG’s evaluation covers the programme’s operations from its inception in 2005 through FY2012. The programme grew from a $500 million annual commitment to $5 billion in FY12. It accounted for 39 percent of total IFC commitments and has low costs—it accounted for 2.4 percent of IFC’s capital use and 1.2 percent of its staff costs and has had no claims to date. It is profitable as well, although not to the extent originally expected, accounting for 0.6 percent of IFC’s net profit. IEG found that the GTFP has particular additionality among higher-risk countries. In its early years, it was concentrated in these countries, particularly in Africa. During the global crisis, the programme risk-mitigation instrument became relevant in much broader markets. Client feedback on the programme has been positive.
In its evaluation IEG does offer several recommendations to enhance its effectiveness, including on issues of transparency and reporting methods, as well as expanding the share of the programme in needier markets. For development professionals, the lessons in this evaluation can be applied to private sector development situations, particularly mitigation of financing risks in emerging markets.
In economic growth the largest differences between developed and developing economies are in innovation performances. Innovation is critical for growth, but it is increasingly important for addressing major development changes, such as those related to inclusion and sustainability.
Recognising this pattern, many countries are promoting innovation and entrepreneurship. Market and government failures and other bottlenecks impede innovation and entrepreneurship, particularly in developing countries. These countries need to build the capacity to find, absorb, and use new technologies and processes as well as foster entrepreneurs who can take risks, look for finance, and bring new products and processes to market.
The Independent Evaluation Group reviewed the investment portfolio in innovation and entrepreneurship interventions over the past decade across the World Bank Group. IEG found that this investment is substantial, but its effectiveness can be enhanced through broad, systemic efforts. Urgent action is required to enhance coordination, consultation, and linkages on innovation and entrepreneurship initiatives across networks, sectors, and regions.
Another challenge is to develop practical solutions for people who earn less than $2 a day–an issue relevant to middle-income as well as low-income countries. Sustained efforts are required to identify different mechanisms and implementation arrangements that are effective and efficient. Monitoring and evaluation is also important for scale-up of promising interventions and mechanisms to effectively capture and share knowledge within and across the Bank Group.
IEG presented a number of recommendations to strengthen the effectiveness of Bank Group support for innovation and entrepreneurship. They involved implementing a strategic framework that considers the new World Bank Group strategy and provides building blocks, examining alternative approaches for financing early-stage start-ups, and promoting knowledge sharing within the Bank Group in numerous ways.
Both development and business audiences will be interested in this evaluation: the lessons distilled and examples highlighted can be applied in many business contexts.
Recognising this pattern, many countries are promoting innovation and entrepreneurship. Market and government failures and other bottlenecks impede innovation and entrepreneurship, particularly in developing countries. These countries need to build the capacity to find, absorb, and use new technologies and processes as well as foster entrepreneurs who can take risks, look for finance, and bring new products and processes to market.
The Independent Evaluation Group reviewed the investment portfolio in innovation and entrepreneurship interventions over the past decade across the World Bank Group. IEG found that this investment is substantial, but its effectiveness can be enhanced through broad, systemic efforts. Urgent action is required to enhance coordination, consultation, and linkages on innovation and entrepreneurship initiatives across networks, sectors, and regions.
Another challenge is to develop practical solutions for people who earn less than $2 a day–an issue relevant to middle-income as well as low-income countries. Sustained efforts are required to identify different mechanisms and implementation arrangements that are effective and efficient. Monitoring and evaluation is also important for scale-up of promising interventions and mechanisms to effectively capture and share knowledge within and across the Bank Group.
IEG presented a number of recommendations to strengthen the effectiveness of Bank Group support for innovation and entrepreneurship. They involved implementing a strategic framework that considers the new World Bank Group strategy and provides building blocks, examining alternative approaches for financing early-stage start-ups, and promoting knowledge sharing within the Bank Group in numerous ways.
Both development and business audiences will be interested in this evaluation: the lessons distilled and examples highlighted can be applied in many business contexts.