Eye-tracking technology now makes it possible for computers to gather staggering amounts of information about individuals as they use the Internet, and draw hyper-accurate conclusions about our behavior as consumers. As the technology becomes more practical, Senior Fellow John Villasenor discusses its benefits and risks.

Video

• Eye-Tracking Technology and Digital Privacy

Audio

• @ Brookings Podcast: Eye-Tracking Technology and Digital Privacy

With new technologies for extracting oil and natural gas producing an energy boom throughout North America, Senior Fellow Charles Ebinger sees the potential in hydraulic fracturing or “fracking” to free the continent from dependence on Middle East oil, and even make some progress on curbing sources of air pollution.
 

##1##

Video

• Fracking and Prospects for Energy Security in North America

In many African countries, women still cannot own land or resources, a significant barrier to their ability to start businesses and take advantage of the continent’s economic potential.  Fellow Anne Kamau explores their plight.

Video

• Challenges for Women in the African Economy

Partisan gridlock and political extremism threaten to tear down the pillars of public policy and render the U.S. government utterly dysfunctional, argues Senior Fellow Thomas Mann, co-author with Norman Ornstein—resident scholar at the American Enterprise Institute—of It’s Even Worse Than It Looks (Basic Books, 2012). He argues that a flood of super-PAC money, negative ads and cowed mainstream news media are contributing to the problem.

Video

• Mann: Blame for Gridlock Is Squarely on Republican Party

A little-known yearly survey conducted by the Census Bureau gathers a wealth of information that helps guide everything from investments in highways and schools to business decisions about building supermarkets and department stores. The American Community Survey gathers neighborhood-level data that isn’t collected in the decennial Census, providing invaluable information about issues from immigration to income levels. Losing that ground-level view of the changes in American life year after year would be disastrous, says Senior Fellow William Frey.

Video

• William Frey: Why the American Community Survey Is Important

Year after year, federal spending on poverty programs has been going up, but we still see more and more people who have no margin to guard against unexpected expenses or job loss. At the same time, for different reasons, Americans who are not impoverished have seen their wealth decline sharply. Expert Ron Haskins, co-director of the Center on Children and Families, says the problems are growing deeper, despite increased federal spending on programs to assist the poor. Haskins says everyone must sacrifice, but also says, that people in general, who finish high school, get a job, and get married and delay having children until age 21 are better off.

Video

• Solutions to Poverty's Rise in America

Climate change and the search for resources have turned the Arctic into an emerging market and an important trade route. Senior Fellow Bruce Jones, director of the Managing Global Order project, says that the top of the world poses possibilities for international tensions among interested nations claiming sovereignty, but at least so far, no serious conflicts have emerged.

Video

• Bruce Jones: Arctic Region’s Importance for Trade, Transit and Energy Is Increasing

The increasing diversification of news media—from online versions of major newspapers to political bloggers, to 24-hour cable news to social media—plus the profession’s changing economics have caused the balance of power between political reporters and presidential candidates to change. Stephen Hess, senior fellow emeritus, says our very good, well-trained reporters are “almost dangerous” to presidential candidates who are trying to stay on message. Thus, says Hess, the way the press covers campaigns has changed as well, and not for the better.

Video

• Stephen Hess: The Changing Balance of Power in Presidential Campaign Reporting

From the Arab-Israeli conflict, to the paradigm shift of the Arab Spring, to attacks on U.S. government personnel in Egypt and Libya, to the potentially explosive situation in Syria--events in the greater Middle East region continue to resonate here at home. In a recent study, “Americans on the Middle East,” Nonresident Senior Fellow Shibley Telhami finds that Americans have a great understanding and concern about Middle East events. Learn more about these findings in this episode of @ Brookings.

Video

• Shibley Telhami: What Americans Think about the Middle East

In this post-election season and with a fiscal cliff looming, states and metros have begun the work of meeting their many challenges. They’re implementing game-changing initiatives to create jobs and restructure their economies for the long haul. The federal government needs to take notice and get on board note, Metropolitan Program policy experts Bruce Katz and Mark Muro as they urge a move for remaking our federalism and renewing the economy. Katz and Muro explain in this episode of @ Brookings.

Video

• Bruce Katz and Mark Muro: Remaking Federalism and Renewing the Economy

Following U.S. troop withdrawal in 2014, Afghanistan faces an uncertain future. Its fate could be compromised or even commanded by war lords, terrorists or corrupt government officials. Fellow Vanda Felbab-Brown spent time on the ground observing events and talking to a mix of Afghans from high ranking officials to village elders, to merchants to the person on the street. In this four-part video series based on her book, “Aspiration and Ambivalence: Strategies and Realities of Counterinsurgency and State-Building in Afghanistan,” Felbab-Brown offers analysis on an Afghanistan in flux.

Video

• Vanda Felbab-Brown: Counterinsurgency and State-Building in Afghanistan after 2014
• Vanda Felbab-Brown: The Afghan People Simply Want to Live and Thrive
• Vanda Felbab-Brown: Pakistan Plays a Significant Role in Afghanistan's Future
• Vanda Felbab-Brown: The Choices the U.S. Makes Will Largely Determine Afghanistan's Future

The civil war tearing through Syria is worsened by a growing tide of refugees and displaced persons along with an escalating humanitarian crisis. Food shortages, a lack of housing and adequate health care are additional burdens that many Syrians now face. Senior Fellow and Brookings-LSE Project on Internal Dispacement Co-Director Elizabeth Ferris examines the cost of war in Syria in this episode of @ Brookings.

Video

• Elizabeth Ferris: Syria’s Escalating Humanitarian Crisis

The horrible tragedy in Orlando is rightly focusing attention on the threat posed by lone wolf terrorists—that is, radicalized individuals or couples who carry out attacks like those in Boston, San Bernardino, and Orlando. Yet there is a greater threat to the national well-being from small groups of terrorists, we can call them wolf packs, that could bring much greater carnage. If America's Muslim community is scapegoated and ostracized, the wolf pack threat will grow.

The idea of lone wolf attacks goes back well before the emergence of the self-styled Islamic State two years ago. The New Mexico-born radical Anwar al Awlaki urged individual Muslim Americans to carry out mass casualty attacks years ago. His English-language web magazine “Inspire” was the vehicle for self-radicalized extremists to learn how to make bombs and conduct violent jihad. The Fort Hood shooter was his disciple. Awlaki's message is still virulent despite his death by drone in Yemen in 2011.

More recently, Hamza bin Laden, the son of al-Qaida's founder, has issued two messages from his hide-out in Pakistan urging lone wolf attacks in Europe, America and Israel. The first came out a year ago and included the endorsement of Ayman Zawahiri. The second came out last month.

The Orlando shooter paid little attention apparently to the differences between radical Islamic organizations. He reportedly praised the Shiite Hezbollah, Sunni al-Qaida, and the Islamic State. The message of jihad is so pervasive now that has become a part of the global dialogue.

As deadly as the Orlando shooting was, it is not hard to envision worse given the horrific attacks in Paris and Brussels this year. Organized conspiracies of a relatively small number of extremists prepared to die in their attacks can do far more damage than individuals or couples.

Organized conspiracies of a relatively small number of extremists prepared to die in their attacks can do far more damage than individuals or couples.

The role model of a wolf pack attack was the November 26, 2008 attack in Mumbai. A pack of ten well-armed and trained Pakistani terrorists attacked simultaneously and sequentially luxury hotels, fancy restaurants, a train station, and a Jewish hostel. Westerners and Israelis were singled out for attack but the majority of victims were innocent Indians. The operation even had a secure command post back in Pakistan where the terrorist handlers from Lashkar e-Tayyeba and Pakistani intelligence coordinated the plot by cell phones.

The Paris attacks imitated many of the Mumbai tactics and added suicide vests to the mix. The Islamic State took the Mumbai model to Europe. It no longer required a foreign base like Pakistan, the conspiracy operated in the depressed Islamic slums of Brussels and Paris where the security services had few sources. 

The closest to a wolf pack attack in the United States was an al-Qaida plot in 2009 to attack the New York City subway system. That was foiled because our intelligence services detected the conspiracy. 

Islamophobia is the path to more catastrophic violence.

By definition, conspiracies involving even small numbers of plotters are easier to detect than lone wolves. The first line of detection is the local Muslim community where the plot is hatched. When local communities become angry pockets separated from the rest of the nation, it is much harder to find and foil conspiracies. In Europe, unfortunately, there are many such disaffected Islamic communities. America doesn't have them, at least not yet.

That fastest and surest route to wolf pack conspiracies in America is to scapegoat all Muslims and create an angry jihadist sub-culture. That is why calls for prohibiting Muslim travel to America are so dangerous and counter-productive. It will inevitably create a backlash. Islamophobia is the path to more catastrophic violence.

"At the end of the day, we need to remember that Daesh is more a product of the civil wars than it is a cause of them. And the way that we’re behaving is we’re treating it as the cause.  And the problem is that in places like Syria, in Iraq, potentially in Libya, we are mounting these military campaigns to destroy Daesh and we’re not doing anything about the underlying civil wars.  And the real danger there is—we have a brilliant military and they may very well succeed in destroying Daesh—but if we haven’t dealt with the underlying civil wars, we’ll have Son of Daesh a year later." – Ken Pollack

“Part of the problem is how we want the U.S. to be more engaged and more involved and what that requires in practice. We have to be honest about a different kind of American role in the Middle East. It means committing considerable economic and political resources to this region of the world that a lot of Americans are quite frankly sick of… There is this aspect of nation-building that is in part what we have to do in the Middle East, help these countries rebuild, but we can’t do that on the cheap. We can’t do that with this relatively hands off approach.” – Shadi Hamid

In this episode of “Intersections,” Kenneth Pollack, senior fellow in the Center for Middle East Policy and Shadi Hamid, senior fellow in the Project on U.S. Relations with the Islamic World and author of "Islamic Exceptionalism: How the Struggle over Islam is Reshaping the World," discuss the current state of upheaval in the Middle East, the Arab Spring, and the political durability of Islamist movements in the region. They also explain their ideas on how and why the United States should change its approach to the Middle East and areas of potential improvement for U.S. foreign policy in the region. 

Show Notes

Fight or flight: America’s choice in the Middle East

Security and public order

Islamists on Islamism today

Temptations of Power: Islamists & Illiberal Democracy in a New Middle East

Ending the Middle East’s civil wars

A Rage for Order: The Middle East in turmoil, from Tahrir Square to ISIS

Building a better Syrian opposition army: How and why

With thanks to audio engineer and producer Zack Kulzer, Mark Hoelscher, Carisa Nietsche, Sara Abdel-Rahim, Eric Abalahin, Fred Dews and Richard Fawal.

Subscribe to the Intersections on iTunes, and send feedback email to intersections@brookings.edu.

The terrible attack in Nice on July 14—Bastille Day—saddened us all. For a country that has done so much historically to promote democracy and human rights at home and abroad, France is paying a terrible and unfair price, even more than most countries. My colleagues Will McCants and Chris Meserole have carefully documented the toll that France, and certain other Francophone countries like Belgium, have suffered in recent years from global terrorism. It is heart wrenching.

From what we know so far, the attack was carried out by a deeply distraught, potentially deranged, and in any case extremely brutal local man from Nice of Tunisian descent and French nationality. Marital problems, the recent loss of his job, and a general sense of personal unhappiness seem to have contributed to the state of mind that led him to commit this heinous atrocity. Perhaps we will soon learn that ISIS, directly or indirectly, inspired the attack in one way or another as well. My colleague Dan Byman has already tapped into his deep expertise about terrorism to remind us that ISIS had in fact encouraged ramming attacks with vehicles before, even if the actual manifestation of such tactics in this case was mostly new. 

This attack will again raise the question: Why France? On this point, I do have a somewhat different take than some of my colleagues. The argument that France has partly brought these tragedies upon itself—perhaps because of its policies of secularism and in particular its limitations on when and where women can wear the veil in France—strikes me as unpersuasive. Its logical policy implications are also potentially disturbing, because if interpreted wrongly, it could lead to a debate on whether France should modify such policies so as to make itself less vulnerable to terrorism. That outcome, even if unintended, could dance very close to the line of encouraging appeasement of heinous acts of violence with policy changes that run counter to much of what French culture and society would otherwise favor. So I feel the need to push back.

Here are some of the arguments, as I see them, against blaming French culture or policy for this recent string of horrible attacks including the Charlie Hebdo massacre, the November 2015 mass shootings in Paris, and the Nice tragedy (as well as recent attacks in Belgium):

• Starting with the simplest point, we still do not know much about the perpetrator of the Nice killings. From what we do surmise so far, personal problems appear to be largely at the root of the violence—different from, but not entirely unlike, the case with the Orlando shooter, Omar Mateen.
• We need to be careful about drawing implications from a small number of major attacks. Since 2000, there have also been major attacks in the Western world by extremist jihadis or takfiris in New York, Washington, Spain, London, San Bernardino, Orlando, and Russia. None of these are Francophone. Even Belgium is itself a mixed country, linguistically and culturally.
• Partly for reasons of geography, as well as history, France does face a larger problem than some other European countries of individuals leaving its country to go to Syria or Iraq to fight for ISIS, and then returning. But it is hardly unique in the scale of this problem.
• Continental Europe has a specific additional problem that is not as widely shared in the United Kingdom or the United States: Its criminal networks largely overlap with its extremist and/or terrorist networks. This point may be irrelevant to the Nice attack, but more widely, extremists in France or Belgium can make use of illicit channels for moving people, money, and weapons that are less available to would-be jihadis in places like the U.K. (where the criminal networks have more of a Caribbean and sub-Saharan African character, meaning they overlap less with extremist networks).
• Of course, the greatest numbers of terrorist attacks by Muslim extremists occur in the broader Muslim world, with Muslims as the primary victims—from Iraq and Syria to Libya and Yemen and Somalia to South Asia. French domestic policies have no bearing on these, of course.

There is no doubt that good work by counterterrorism and intelligence forces is crucial to preventing future attacks. France has done well in this regard—though it surely can do better, and it is surely trying to get better. There is also no doubt that promoting social cohesion in a broad sense is a worthy goal. But I would hesitate, personally, to attribute any apparent trend line in major attacks in the West to a particular policy of a country like France—especially when the latter is in fact doing much to seek to build bridges, as a matter of national policy, with Muslims at home and abroad. 

There is much more to do in promoting social cohesion, to be sure, even here in America (though our own problems probably center more on race than on religion at the moment). But the Nice attacker almost assuredly didn’t attack because his estranged wife couldn’t wear a veil in the manner and/or places she wanted. At a moment like this in particular, I disagree with insinuations to the contrary.

Event Information

On July 20 and 21, defense ministers from several nations will gather in Washington, D.C. at the invitation of U.S. Secretary of Defense Ash Carter. The meeting will bring together representatives from countries working to confront and defeat the Islamic State (or ISIL). French Defense Minister Jean-Yves Le Drian will be among those at the summit discussing how to accelerate long-term efforts to fight ISIL in Iraq and Syria. The close relationship between France and the United States has provided a solid base for security cooperation for decades, and in recent years, France has become one of America’s strongest allies in fighting terrorism and a prominent member of the international coalition to defeat ISIL.

On July 20, the Foreign Policy program at Brookings hosted Minister Le Drian for a discussion on French and U.S. cooperation as the two countries face multiple transnational security threats. Since becoming France’s defense minister in 2012, Le Drian has had to address numerous new security crises emerging from Africa, the Middle East, and within Europe itself. France faced horrific terrorist attacks on its own soil in January and November 2015 and remains under a state of emergency with its armed forces playing an active role in maintaining security both at home and abroad. Le Drian recently authored “Qui est l’ennemi?” (“Who is the enemy?”, Editions du Cerf, May 2016), defining a comprehensive strategy to address numerous current threats.

Join the conversation on Twitter using #USFrance

Video

• Introduction and featured speaker
• Discussion
• Introduction et conférencier invité
• Débat

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• 20160720_france_defense_transcript

France has been struck by an unprecedented three terror attacks in the last 18 months. In what’s called Operation Sentinelle, 13,000 French military personnel now patrol streets and protect key sites across the country, assisting police and other security agencies. “The fact that the armed forces are visible,” said French Defense Minister Jean-Yves Le Drian at a Brookings event on July 20, “help to reassure the French people that they are safe both at home and abroad.”

Do the challenges facing France today mean that it should reduce its engagement overseas, focusing instead on security at home? Le Drian doesn’t think so, and I agree. In a new book titled "Who is the Enemy?," he particularly emphasizes the multifaceted ISIS threat. As he said at Brookings: 

"Every war [has] two enemies…[Today’s] war [with ISIS] also sets in place two concepts of the “enemy” that are radically different: From a strategic point of view, we are dealing with a proto-state; at the heart of this entity, there is a terrorist army." 

It only further complicates matters, of course, that France faces ISIS threats on several fronts: in Syria and Iraq, on the one hand, and also on its own territory. This, Le Drian stressed, means “we must seek coherence in our military action.” It also helps explain why France remains one of the most active countries in the fight against the so-called Islamic State, as well as other extremist groups in the Middle East and in sub-Saharan Africa. In 2013 and 2014, France intervened in Mali in order to prevent jihadi groups from taking over the country. The French military also has a presence in Djibouti, Lebanon, Côte d’Ivoire, the Central African Republic, Gabon, Senegal, as well as in the Pacific (in French Polynesia and New Caledonia)—not to mention Syria, where France uses the Mediterranean Sea-based Charles de Gaulle aircraft carrier to strike ISIS targets. 

France is not a warmongering country—rather, it is responding to the fact that overseas threats come to it. Although it remains unclear to what extent the Nice attacker had connections with foreign terrorist networks, it has been established that the November 2015 Paris attacks were planned and orchestrated from Syria. This, among other considerations, has prompted France to engage further in Iraq and Syria. The rationale, as Minister Le Drian explained, is: 

“[T]rading our peace by reducing our military involvement doesn’t make sense. The more we let ISIS consolidate its presence on the Middle East, the more it will gather resources, attract fighters, and plan more attacks against us.” 

Team player

French policy isn’t just about ensuring its own security—rather, its many contributions are integrated within global efforts, including U.S.-led ones. As Le Drian said at Brookings: “I am convinced that the French-American relationship is stronger and better than ever.” France is a prominent participant of the 66-member international coalition against ISIS, and in that capacity participated in the first joint meeting of that group’s foreign and defense ministers in Washington this month. 

France remains a key member of the joint military operation Inherent Resolve in Iraq and Syria, which has damaged or destroyed over 26,0000 ISIS-related targets since August 2014. The Charles de Gaulle carrier—with 26 aircrafts on board—has been an essential part of that coalition mission. Following specific instructions from U.S. Defense Secretary Ash Carter and his counterpart in Paris, Minister Le Drian, French and U.S. intelligence agencies cooperate closely in intelligence-sharing. And just last week, President François Hollande announced that France will soon be supplying artillery to Iraq to support its fight against ISIS. Beyond the Iraq-Syria theater, France is cooperating with the United States and other partners in Libya, another country that is both a victim and source of extremist threats. 

The French Defense Ministry’s efforts to double-down on protecting French citizens within France, therefore, has not reduced its overseas role. Particularly now that the United Kingdom will leave the European Union, France’s military role has never been so important. France—along with Germany, which recently suggested it would raise its defense spending significantly—should continue to play a leading role as one the top defense actors in the West. 

Abstract

Scaling up of development interventions is much debated today as a way to improve their impact and effectiveness. Based on a review of scaling up literature and practice, this paper develops a framework for the key dynamics that allow the scaling up process to happen. The authors explore the possible approaches and paths to scaling up, the drivers of expansion and of replication, the space that has to be created for interventions to grow, and the role of evaluation and of careful planning and implementation. They draw a number of lessons for the development analyst and practitioner. More than anything else, scaling up is about political and organizational leadership, about vision, values and mindset, and about incentives and accountability—all oriented to make scaling up a central element of individual, institutional, national and international development efforts. The paper concludes by highlighting some implications for aid and aid donors.

An annotated bibliography of the literature on scaling up and development aid effectiveness was created by Oksana Pidufala to supplement this working paper. Read more »

Downloads

• Download

EXECUTIVE SUMMARY

Fragile states present one of the greatest challenges to global development and poverty reduction. Despite much new learning that has emerged from within the development community in recent years, understanding of how to address fragility remains modest. There is growing recognition that donor engagement in fragile states must look beyond the confines of the traditional aid effectiveness agenda if it is to achieve its intended objectives, which include statebuilding, meeting the needs of citizens, and managing risk more effectively. Current approaches are constrained by relying heavily on small-scale interventions, are weakened by poor coordination and volatility, and struggle to promote an appropriate role for the recipient state.

Scaling up (i.e., the expansion, replication, adaption and sustaining of successful policies and programs in space and over time to reach a greater number of people) is highly relevant to fragile settings, both as an objective and as a strategic approach to development. As an objective, it reinforces the logic that the scale of the challenges in fragile states demands interventions that are commensurate in purpose and equal to the task. As a strategy, it encourages donors to identify and leverage successes, and to integrate institutional development more explicitly into projects and programs. In addition, scaling up can assist donors in addressing the priority areas of improved project design and implementation, sustainability and effective risk management.

Successful scaling up in fragile states almost certainly occurs less often than is possible and does not always involve a systematic approach. Donors should therefore look to more systematically pursue scaling up in fragile states and evaluate their performance with specific reference to this objective. This can be done by incorporating relevant elements of a scaling up framework into operational policies, from strategy development through to program design and monitoring.

Contrary to expectations, there are compelling examples of successful scaling up in fragile states. While the conditions prevailing in fragile states create serious obstacles in terms of “drivers” (the forces that push the scaling up process forward) and “spaces” (the opportunities that need to be created, or potential obstacles that need to be removed for interventions to grow), and in terms of the operational modalities of donors, these can be overcome through the careful design and delivery of programs with a clear focus on creating scaling up pathways, and through close partnership and sustained engagement of governments, communities and foreign partners.

Case study evidence suggests that the pathways taken to reach scale in fragile states demand different approaches by donors. Donors need to adopt greater selectivity in determining which areas or sectors for scaling up are justified—a strategy that has also assisted some donors in managing risk. More investment and time are required in upfront analysis and building the evidence for successful scaling up pathways. In some cases, donors require longer time horizons to achieve scale, although demand from government or beneficiaries has sometimes forced donors to move immediately to scale, allowing little or no time for piloting. Regardless of the pace of scaling up, donors that were most successful were engaged early and then remained engaged, often far beyond the replication phase of scaling up, to increase the likelihood of interventions being sustained. Other common characteristics of successful scaling up were simple project design and a focus on the institutional aspects of the scaling up pathway.

Case studies also point to the crucial role of drivers in moving the scaling up process forward in fragile states. Proven ideas and practical models have often been picked up in fragile states, contrary to the expectation that actors may be less responsive to recognizing and acting on the utility of promising results. Leaders undoubtedly have a role to play in supporting scaling up, although there are clear dangers that must be avoided, including avoiding the perception that donors are picking (political) winners by nominating leaders, and tying the survival of projects too closely to the fortunes of a leader’s political career. Incentives were found to be one of the most important drivers in fragile states, and there is a good case to be made for donors introducing new inducements, greater transparency or similar reforms to strengthen the role incentives play. Finally, and in contrast to the standard scaling up framework, community demand was found to be an important driver in many fragile states, both in demanding the expansion of small-scale projects and by facilitating the community’s own resources to support the scaling up process.

The greatest challenge to scaling up in fragile states is the limited spaces these environments provide. This is especially the case in respect to those spaces which concern aspects of governance: political, institutional and policy spaces. When working in fragile states, donors must recognize that spaces for scaling up are almost always more constrained, but look for ways to expand upon them. Some of the most successful examples of scaling up used creative approaches to build space quickly or used existing capacity to the fullest possible extent. Also relevant are the lessons of robust analysis, greater realism and cost control. The case studies confirm the importance of two additional spaces in fragile states. For example, security space often imposed horizontal obstacles to scaling up which could not realistically be overcome while ownership space served as a good indication of the perceived legitimacy of the scaling up process and the likelihood that interventions would be sustained longer term.

Case studies also affirm the importance of emphasizing robust project design and implementation, and the close linkages between the scaling up agenda and the role of risk management and sustainability in fragile states. While sustainability presented a significant problem for many of the projects and programs reviewed, a more focused approach around scaling up may assist donors in addressing sustainability concerns. This would entail adopting a longer-term perspective beyond the immediate confines of any individual project, looking for available drivers and supportive spaces, and focusing on effective implementation and consistent monitoring and evaluation (M&E). Any intervention introduced on a small scale that scores well in sustainability serves as a possible candidate for scaling up.

Similarly, many of the methods used by donors for managing risk—an emphasis on analysis, scenario planning, realism and making use of specialized aid instruments—are equally relevant for supporting scaling up in fragile countries. A persuasive argument can be made that the adoption of a more explicit scaling up approach by donors can form part of a risk management strategy in fragile states. Scaling up can enable donors to more ambitiously tackle development risks without allowing institutional and project risks to grow unchecked. Ultimately, a donor approach that combines good risk management and scaling up requires strong leadership and well-aligned incentives.

Downloads

• Download the full paper

A key objective of the United Nations Development Programme (UNDP) is to assist its member countries in meeting the Millennium Development Goals (MDGs). UNDP pursues this objective in various ways, including through analysis and advice to governments on the progress towards the MDGs (such as support for the preparation and monitoring Poverty Reduction Strategies, or PRSs, in poor countries), assistance for capacity building, and financial and technical support for the preparation and implementation of development programs.

The challenge of achieving the MDGs remains daunting in many countries, including Tajikistan. To do so will require that all development partners, i.e., the government, civil society, private business and donors, make every effort to scale up successful development interventions. Scaling up refers to “expanding, adapting and sustaining successful policies, programs and projects on different places and over time to reach a greater number of people.” Interventions that are successful as pilots but are not scaled up will create localized benefits for a small number of beneficiaries, but they will fail to contribute significantly to close the MDG gap.

This paper aims to assess whether and how well UNDP is supporting scaling up in its development programs in Tajikistan. While the principal purpose of this assessment was to assist the UNDP country program director and his team in Tajikistan in their scaling up efforts, it also contributes to the overall growing body of evidence on the scaling up of development interventions worldwide.

Downloads

• Download Full Paper

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

• An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
• An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
• An evaluation of evaluations should assess:
  o How effectively impact evaluations are used;
  o How scaling up of successful interventions is treated;
  o How the experience of other comparable institutions is utilized;
  o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
  o Whether and how the governance of the institution is evaluated; and
  o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.