Eye-tracking technology now makes it possible for computers to gather staggering amounts of information about individuals as they use the Internet, and draw hyper-accurate conclusions about our behavior as consumers. As the technology becomes more practical, Senior Fellow John Villasenor discusses its benefits and risks.

Video

• Eye-Tracking Technology and Digital Privacy

Audio

• @ Brookings Podcast: Eye-Tracking Technology and Digital Privacy

With new technologies for extracting oil and natural gas producing an energy boom throughout North America, Senior Fellow Charles Ebinger sees the potential in hydraulic fracturing or “fracking” to free the continent from dependence on Middle East oil, and even make some progress on curbing sources of air pollution.
 

##1##

Video

• Fracking and Prospects for Energy Security in North America

Baltimore provides a prime example of how metropolitan areas around the nation are turning to clean, green industries as a source of vibrant, sustainable growth. Expert Jennifer Vey outlines how such communities can identify their assets and capitalize on them to revitalize their economies.

Video

• Baltimore as a Case Study in Metro Economic Recovery

Audio

• @ Brookings Podcast: Baltimore as a Case Study in Metro Economic Recovery

In many African countries, women still cannot own land or resources, a significant barrier to their ability to start businesses and take advantage of the continent’s economic potential.  Fellow Anne Kamau explores their plight.

Video

• Challenges for Women in the African Economy

Partisan gridlock and political extremism threaten to tear down the pillars of public policy and render the U.S. government utterly dysfunctional, argues Senior Fellow Thomas Mann, co-author with Norman Ornstein—resident scholar at the American Enterprise Institute—of It’s Even Worse Than It Looks (Basic Books, 2012). He argues that a flood of super-PAC money, negative ads and cowed mainstream news media are contributing to the problem.

Video

• Mann: Blame for Gridlock Is Squarely on Republican Party

More than a decade after the start of the war in Afghanistan, America continues to face significant challenges in the Middle East. While news of U.S. struggles often dominate foreign policy discussions, Senior Fellow Bruce Riedel says it is important to remember that the United States is also making progress in the region. From the death of Osama bin Laden to an agreement on the use of Afghan military bases for U.S. counterterrorism operations, America is learning from its past mistakes and using these lessons to guide its response to the Arab Spring.

Video

• We Shouldn't Lose Sight of the Positive Developments in the Middle East

Drought, low agriculture productivity and a cascade of bad policy choices have put millions in the Horn of Africa and the Sahel at risk for starvation. Senior Fellow Mwangi Kimenyi, director of the Africa Growth Initiative, says the consequences will haunt the continent for decades to come.

Video

• Mwangi S. Kimenyi: Food Insecurity in Africa Can Be Solved

A little-known yearly survey conducted by the Census Bureau gathers a wealth of information that helps guide everything from investments in highways and schools to business decisions about building supermarkets and department stores. The American Community Survey gathers neighborhood-level data that isn’t collected in the decennial Census, providing invaluable information about issues from immigration to income levels. Losing that ground-level view of the changes in American life year after year would be disastrous, says Senior Fellow William Frey.

Video

• William Frey: Why the American Community Survey Is Important

In his book, Our Divided Political Heart, Senior Fellow E.J. Dionne, Jr. writes that the current deep political divide plaguing American politics has its roots in a fundamental misunderstanding of our founding principles and the strong role our forebears saw for government in protecting individual liberty.

Video

• E.J. Dionne, Jr.: Founding Fathers Believed Common Good Protected Individual Liberty

Year after year, federal spending on poverty programs has been going up, but we still see more and more people who have no margin to guard against unexpected expenses or job loss. At the same time, for different reasons, Americans who are not impoverished have seen their wealth decline sharply. Expert Ron Haskins, co-director of the Center on Children and Families, says the problems are growing deeper, despite increased federal spending on programs to assist the poor. Haskins says everyone must sacrifice, but also says, that people in general, who finish high school, get a job, and get married and delay having children until age 21 are better off.

Video

• Solutions to Poverty's Rise in America

Climate change and the search for resources have turned the Arctic into an emerging market and an important trade route. Senior Fellow Bruce Jones, director of the Managing Global Order project, says that the top of the world poses possibilities for international tensions among interested nations claiming sovereignty, but at least so far, no serious conflicts have emerged.

Video

• Bruce Jones: Arctic Region’s Importance for Trade, Transit and Energy Is Increasing

Emerging economies may chafe at international agreements calling for sustainable development, but Nonresident Fellow Nathan Hultman says many governments are putting plans for sustainability and green innovation in place out of self-interest, and cooperating with neighbors across the globe.

Video

• Nathan Hultman: Global Progress in Sustainable Development

The increasing diversification of news media—from online versions of major newspapers to political bloggers, to 24-hour cable news to social media—plus the profession’s changing economics have caused the balance of power between political reporters and presidential candidates to change. Stephen Hess, senior fellow emeritus, says our very good, well-trained reporters are “almost dangerous” to presidential candidates who are trying to stay on message. Thus, says Hess, the way the press covers campaigns has changed as well, and not for the better.

Video

• Stephen Hess: The Changing Balance of Power in Presidential Campaign Reporting

While falling off the "fiscal cliff" (of automatic spending cuts and tax increases if Congress fails to act) could hurt the economy, expert William Gale says the actual result, if coupled with a temporary economic stimulus, would be greater incentives to make a better long-term budget deal.

Video

• William Gale: Combine Going Over the Fiscal Cliff with a Stimulus

From the Arab-Israeli conflict, to the paradigm shift of the Arab Spring, to attacks on U.S. government personnel in Egypt and Libya, to the potentially explosive situation in Syria--events in the greater Middle East region continue to resonate here at home. In a recent study, “Americans on the Middle East,” Nonresident Senior Fellow Shibley Telhami finds that Americans have a great understanding and concern about Middle East events. Learn more about these findings in this episode of @ Brookings.

Video

• Shibley Telhami: What Americans Think about the Middle East

In this post-election season and with a fiscal cliff looming, states and metros have begun the work of meeting their many challenges. They’re implementing game-changing initiatives to create jobs and restructure their economies for the long haul. The federal government needs to take notice and get on board note, Metropolitan Program policy experts Bruce Katz and Mark Muro as they urge a move for remaking our federalism and renewing the economy. Katz and Muro explain in this episode of @ Brookings.

Video

• Bruce Katz and Mark Muro: Remaking Federalism and Renewing the Economy

Following U.S. troop withdrawal in 2014, Afghanistan faces an uncertain future. Its fate could be compromised or even commanded by war lords, terrorists or corrupt government officials. Fellow Vanda Felbab-Brown spent time on the ground observing events and talking to a mix of Afghans from high ranking officials to village elders, to merchants to the person on the street. In this four-part video series based on her book, “Aspiration and Ambivalence: Strategies and Realities of Counterinsurgency and State-Building in Afghanistan,” Felbab-Brown offers analysis on an Afghanistan in flux.

Video

• Vanda Felbab-Brown: Counterinsurgency and State-Building in Afghanistan after 2014
• Vanda Felbab-Brown: The Afghan People Simply Want to Live and Thrive
• Vanda Felbab-Brown: Pakistan Plays a Significant Role in Afghanistan's Future
• Vanda Felbab-Brown: The Choices the U.S. Makes Will Largely Determine Afghanistan's Future

The civil war tearing through Syria is worsened by a growing tide of refugees and displaced persons along with an escalating humanitarian crisis. Food shortages, a lack of housing and adequate health care are additional burdens that many Syrians now face. Senior Fellow and Brookings-LSE Project on Internal Dispacement Co-Director Elizabeth Ferris examines the cost of war in Syria in this episode of @ Brookings.

Video

• Elizabeth Ferris: Syria’s Escalating Humanitarian Crisis

Despite widespread agreement that the federal budget is on an unsustainable path, there is also widespread disagreement about what should be done. The Hamilton Project asked leading experts from a variety of backgrounds—the policy world, academia, and the private sector, and from both sides of the political aisle—to develop and share their ideas for addressing the deficit. The proposals will be released at two events scheduled for February 22 and February 26. In a dialogue previewing those events, Hamilton Project Director Michael Greenstone and Policy Director Adam Looney discuss some of the key ideas offered by the experts.

Greenstone stresses that the goal of the papers is to move beyond the fights and disagreements between President Obama and Congress and to provide some of the poetry, or some of the details, on how government might run better. The papers will also be featured in a book, 15 Ways to Rethink the Federal Budget, and will take on a wide-ranging set of topics, including immigration, transportation, health care, defense spending, and tax expenditures.

Video

• Michael Greenstone and Adam Looney: These Authors Have Drafted Policies That Help the Budget and Provide Economic Benefits

Although there has been some progress in forming a national unity government in Libya, “unity” is a rather inapplicable word for the country. In reality, friction between various political actors remains high. Ultimately, perhaps a form of disunity—confederation, rather than centralization—is the best model for Libya.

Libyan politics: A primer

During the summer of 2014, the Libyan leadership, after an initial hint of cooperation, split into two governments: 

• One, headquartered in Tobruk and based on a secular matrix, was recognized internationally. It received support from the House of Representatives and was abetted by General Khalifa Haftar and his so-called National Libyan Army. Externally, Egypt, the United Arab Emirates, and Russia have supported this government because of its anti-Islamist ideology. In May 2014, Haftar launched "Operation Dignity" against the Islamist militias, supported by the Zintan brigades (consisting of the Civic, al-Sawaiq, and al-Qaaqa brigades), and the militias coming from the ethnic minorities of Tebu and Fezzan.
• The other, headquartered in Tripoli, was Islamic in nature. It was supported by the new General National Congress (GNC) and was part of the Libya Dawn group of pro-Islamist militias (which included groups from Misrata, Amazigh, and Tuareg). Qatar, Sudan, and Turkey have supported this government for different reasons, including to earn a more prominent place on the global stage or to support the Muslim Brotherhood. 

But it gets more complicated, since it wasn’t just the Tobruk- and Tripoli-based governments that competed to fill the power vacuum post-Gadhafi. The constellation of militias and brigades has changed continuously. There are Salafist groups such as:

• Ansar al-Sharia Libya (or ASL, located between Benghazi and Derna); 
• Muhammad Jamal Network (between Benghazi and Derna);
• Al-Murabitun (in the southeast, around Ghat, Ubari, Tasawah, and Murzuq);
• Al-Qaida in the Islamic Maghreb (or AQIM, in the southwest and northeast of Libya); and
• Ansar al-Sharia Tunisia (or AST, located between Derna and Ajdabiya). 

Then in 2015, an Islamic State (or ISIS) cell—made up of about 3,000 Tunisians, Yemenis, Algerians, and Libyans, especially former supporters of the Gadhafi regime and members of Ansar al-Sharia—began to take hold in the city of Sirte, Gadhafi's hometown. Sirte is in an oil-rich desert area with tremendous strategic value, lying between the two regions of Tripolitania and Cyrenaica. And Misratan militias treated Sirte ruthlessly after Gadhafi’s fall, prompting many locals to welcome ISIS. So it was no accident that ISIS chose that spot, or that it stepped into the Libyan power vacuum more broadly: Libya is strategically important for eventually expanding across North Africa; it’s a launching point for criminal trafficking in the Mediterranean; and there is a potential to exploit huge energy resources, as ISIS has done to a degree in Iraq. 

Then in December 2015 in Morocco, the Government of National Unity (GNA) signed an agreement by which Fayez Serraj became prime minister. But General Haftar and the government in Tobruk didn’t support the move, and the security environment across the country is still abysmal. Despite the assurances from United Nations Support Mission in Libya (UNSMIL) Special Envoy Martin Kobler that Libya would achieve stability, Libya is still seriously fragmented. 

Today, the real fight is not even between Cyrenaica and Tripolitania, per se, because the two regions—along with Fezzan—are so divided internally. Serraj was barely allowed to arrive in Tripoli this March, for instance—only thanks to the intervention of the international community. The GNC (with Prime Minister Khalifa Gwell and President Nouri Abusahmain) immediately dubbed Serraj’s cabinet "illegal," but then a month later decided to disband in favor of Serraj’s GNA. The government in Tobruk, led by President of the House of Representatives Aguila Saleh Issa, has still not given his full endorsement.

Bright spots?

In spite of these political frictions, there have been small signs of progress. Foreign ministers from other countries and even the prime minister of Malta have arrived in Tripoli as a sign that the new political situation is formalizing. And while embassies remain closed, there is a sense that things are moving in a positive direction. Given this, in late April Serraj asked the international community to intervene in order to secure oil wells, theoretically protected by Jadran Ibrahim and his Petroleum Facilities Guard (PFG), a powerful allied militia in Tripoli. But while the international community has seemed ready—including the Italian government, which has taken a leadership role—accusations of local weakness and Western meddling complicate the Libyan political arena. 

ISIS, meanwhile, is suffering setbacks, having been attacked in Sirte from the south, west, and east by a collection of GNA forces, Misrata militia brigades, and the PFG. The GNA forces are currently in the center of Sirte, clashing with ISIS and gaining terrain every day. ISIS seems to be weaker than many thought (indicating that estimates of its numbers were wrong) and now may be fleeing south—to Fezzan—where its strategy can be more fluid and less based on territorial control. 

Re-considering the fragmentation problem

The persistent fragmentation in Libya is what is most worrying. Internal divisions are the product of decades of Gadhafi’s reckless governing—he kept his citizens from each other and from the rest of the world and deprived them of any solid governmental or administrative structure that could keep the country stable in the event of a "post-regime" moment. And looking even further back, it’s important to remember that Tripolitania and Cyrenaica were never aligned, even during the two decades of rebellion against Italy. The Italians used the old "divide et impera" (divide and conquer) strategy, digging real "furrows of blood"—in the words of British scholar Edward E. Evans-Pritchard in 1949—between Libyan tribes. 

And today? A serious agreement between the main political factions—the Government of National Unity and the House of Representatives—seems out of reach. Meanwhile, few of the fundamental institutions required for the development and governance of a modern country are in place. Libya has invested little in education, and both corruption and unemployment are off the charts. Despite immense energy resources, the economy is contracting. Oil production has declined from 500,000 barrels per day in 2013 to 300,000 in January 2016, and not because deposits have depleted. And tourism, it goes without saying, isn’t taking place. 

Fayez Serraj, Libyan prime minister-designate under the proposed unity government, attends a meeting with officials of municipal council of Tripoli in Tripoli, Libya. Photo credit: Reuters/Ismail Zitouny.

Instead, there have been thousands of deaths and a massive outflow of refugees. While UNSMIL’s efforts have been commendable, the international community should seriously consider how to do more in Libya. It’s better to devise and implement an intervention plan now than wait for a true emergency in Libya. The international community must think about and articulate a real strategy, not merely implement tactical operations. Given the dramatically deteriorated security situation today, it seems impossible to imagine a non-security related intervention, even in defense of the soldiers called to the simple mission of protecting the new coalition government. 

One approach to consider is helping Libyans build a confederal state, divided into three large regions: Tripolitania, Cyrenaica, and Fezzan (or perhaps more if the Libyan people deem it appropriate). Perhaps it is time that such provinces become more autonomous—following different paths as they choose, based on their unique ethnic, social, religious, and political origins. This is an extreme solution, of course. But it is clear that the international community, which had been so much a part of the Libyan revolution, cannot now permit the failure of Libya as a state.

The paradox of deconstructing to construct, in this case, can work. The long-advocated national-level solution of political unity does not, in fact, seem possible. Instead, a confederation of the three regions built on the original disposition of tribes and natural borders could probably assure a deeper stability. Regional governments could better protect local interests in security, economic reconstruction, and governance. The international community should thus start from the bottom, emphasizing local solutions, supporting local actors, and helping to empower Libyans to choose their leaders at a local level. This is not to rule out a central government someday, but would mean that such a government would be somewhat less influential. It’s an incredibly difficult and long plan, but probably the only one that can work. 

France has been struck by an unprecedented three terror attacks in the last 18 months. In what’s called Operation Sentinelle, 13,000 French military personnel now patrol streets and protect key sites across the country, assisting police and other security agencies. “The fact that the armed forces are visible,” said French Defense Minister Jean-Yves Le Drian at a Brookings event on July 20, “help to reassure the French people that they are safe both at home and abroad.”

Do the challenges facing France today mean that it should reduce its engagement overseas, focusing instead on security at home? Le Drian doesn’t think so, and I agree. In a new book titled "Who is the Enemy?," he particularly emphasizes the multifaceted ISIS threat. As he said at Brookings: 

"Every war [has] two enemies…[Today’s] war [with ISIS] also sets in place two concepts of the “enemy” that are radically different: From a strategic point of view, we are dealing with a proto-state; at the heart of this entity, there is a terrorist army." 

It only further complicates matters, of course, that France faces ISIS threats on several fronts: in Syria and Iraq, on the one hand, and also on its own territory. This, Le Drian stressed, means “we must seek coherence in our military action.” It also helps explain why France remains one of the most active countries in the fight against the so-called Islamic State, as well as other extremist groups in the Middle East and in sub-Saharan Africa. In 2013 and 2014, France intervened in Mali in order to prevent jihadi groups from taking over the country. The French military also has a presence in Djibouti, Lebanon, Côte d’Ivoire, the Central African Republic, Gabon, Senegal, as well as in the Pacific (in French Polynesia and New Caledonia)—not to mention Syria, where France uses the Mediterranean Sea-based Charles de Gaulle aircraft carrier to strike ISIS targets. 

France is not a warmongering country—rather, it is responding to the fact that overseas threats come to it. Although it remains unclear to what extent the Nice attacker had connections with foreign terrorist networks, it has been established that the November 2015 Paris attacks were planned and orchestrated from Syria. This, among other considerations, has prompted France to engage further in Iraq and Syria. The rationale, as Minister Le Drian explained, is: 

“[T]rading our peace by reducing our military involvement doesn’t make sense. The more we let ISIS consolidate its presence on the Middle East, the more it will gather resources, attract fighters, and plan more attacks against us.” 

Team player

French policy isn’t just about ensuring its own security—rather, its many contributions are integrated within global efforts, including U.S.-led ones. As Le Drian said at Brookings: “I am convinced that the French-American relationship is stronger and better than ever.” France is a prominent participant of the 66-member international coalition against ISIS, and in that capacity participated in the first joint meeting of that group’s foreign and defense ministers in Washington this month. 

France remains a key member of the joint military operation Inherent Resolve in Iraq and Syria, which has damaged or destroyed over 26,0000 ISIS-related targets since August 2014. The Charles de Gaulle carrier—with 26 aircrafts on board—has been an essential part of that coalition mission. Following specific instructions from U.S. Defense Secretary Ash Carter and his counterpart in Paris, Minister Le Drian, French and U.S. intelligence agencies cooperate closely in intelligence-sharing. And just last week, President François Hollande announced that France will soon be supplying artillery to Iraq to support its fight against ISIS. Beyond the Iraq-Syria theater, France is cooperating with the United States and other partners in Libya, another country that is both a victim and source of extremist threats. 

The French Defense Ministry’s efforts to double-down on protecting French citizens within France, therefore, has not reduced its overseas role. Particularly now that the United Kingdom will leave the European Union, France’s military role has never been so important. France—along with Germany, which recently suggested it would raise its defense spending significantly—should continue to play a leading role as one the top defense actors in the West. 

Since mid-2014, the world’s attention has been transfixed on the aesthetically shocking actions of ISIS and the threat it poses to regional and international security. However, it is arguably Jabhat al-Nusra in Syria—and perhaps the al-Qaida movement more broadly—that looks more likely to survive over the long term and to threaten local, regional and international security interests. Since its emergence in Syria in late 2011, Jabhat al-Nusra has transformed itself from an unpopular outsider accused of Islamic State in Iraq (ISI)-like brutality towards one of the most powerful armed actors in the Syrian crisis. Moreover, its break away from the ISI in April 2013 set it further down a path of deep integration into the broader Syrian armed opposition in its fight against Bashar Assad’s regime.

Nearly five years after its formation, Jabhat al-Nusra has demonstrated the potential value of its "long game" approach. By adopting a strategy of gradualism, it has socialized populations into first accepting, and then supporting and defending, this al-Qaida-like movement. Jabhat al-Nusra aims to epitomize the realization of al-Qaida’s evolved thinking. It seeks to build localized bases of influence by embedding itself in within popular revolutionary dynamics and, eventually, establish zones of territorial control from which it can launch attacks against the Western world. By establishing a durable presence in Syria and potentially considering separating itself from al-Qaida internationally, Jabhat al-Nusra seeks to realize its long-term vision of establishing Islamic Emirates inside Syria, as components of a future Caliphate.

Jabhat al-Nusra has successfully prepared its surroundings in such a way as to give it an improved chance of surviving in the long-term, despite international counterterrorism efforts. The international community must work to de-escalate the situation in Syria and more forcefully push for a diplomatic settlement so as to prevent the establishment of a longstanding jihadi safe haven. Policy decisions, including expanding support to Syria’s mainstream opposition, encouraging opposition-Kurdish dialogue, and interrupting al-Qaida’s finances, can diminish Jabhat al-Nusra’s chances of survival.

Downloads

• Profiling Jabhat al-Nusra

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

• An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
• An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
• An evaluation of evaluations should assess:
  o How effectively impact evaluations are used;
  o How scaling up of successful interventions is treated;
  o How the experience of other comparable institutions is utilized;
  o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
  o Whether and how the governance of the institution is evaluated; and
  o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.