The civil war tearing through Syria is worsened by a growing tide of refugees and displaced persons along with an escalating humanitarian crisis. Food shortages, a lack of housing and adequate health care are additional burdens that many Syrians now face. Senior Fellow and Brookings-LSE Project on Internal Dispacement Co-Director Elizabeth Ferris examines the cost of war in Syria in this episode of @ Brookings.

Video

• Elizabeth Ferris: Syria’s Escalating Humanitarian Crisis

The terrible attack in Nice on July 14—Bastille Day—saddened us all. For a country that has done so much historically to promote democracy and human rights at home and abroad, France is paying a terrible and unfair price, even more than most countries. My colleagues Will McCants and Chris Meserole have carefully documented the toll that France, and certain other Francophone countries like Belgium, have suffered in recent years from global terrorism. It is heart wrenching.

From what we know so far, the attack was carried out by a deeply distraught, potentially deranged, and in any case extremely brutal local man from Nice of Tunisian descent and French nationality. Marital problems, the recent loss of his job, and a general sense of personal unhappiness seem to have contributed to the state of mind that led him to commit this heinous atrocity. Perhaps we will soon learn that ISIS, directly or indirectly, inspired the attack in one way or another as well. My colleague Dan Byman has already tapped into his deep expertise about terrorism to remind us that ISIS had in fact encouraged ramming attacks with vehicles before, even if the actual manifestation of such tactics in this case was mostly new. 

This attack will again raise the question: Why France? On this point, I do have a somewhat different take than some of my colleagues. The argument that France has partly brought these tragedies upon itself—perhaps because of its policies of secularism and in particular its limitations on when and where women can wear the veil in France—strikes me as unpersuasive. Its logical policy implications are also potentially disturbing, because if interpreted wrongly, it could lead to a debate on whether France should modify such policies so as to make itself less vulnerable to terrorism. That outcome, even if unintended, could dance very close to the line of encouraging appeasement of heinous acts of violence with policy changes that run counter to much of what French culture and society would otherwise favor. So I feel the need to push back.

Here are some of the arguments, as I see them, against blaming French culture or policy for this recent string of horrible attacks including the Charlie Hebdo massacre, the November 2015 mass shootings in Paris, and the Nice tragedy (as well as recent attacks in Belgium):

• Starting with the simplest point, we still do not know much about the perpetrator of the Nice killings. From what we do surmise so far, personal problems appear to be largely at the root of the violence—different from, but not entirely unlike, the case with the Orlando shooter, Omar Mateen.
• We need to be careful about drawing implications from a small number of major attacks. Since 2000, there have also been major attacks in the Western world by extremist jihadis or takfiris in New York, Washington, Spain, London, San Bernardino, Orlando, and Russia. None of these are Francophone. Even Belgium is itself a mixed country, linguistically and culturally.
• Partly for reasons of geography, as well as history, France does face a larger problem than some other European countries of individuals leaving its country to go to Syria or Iraq to fight for ISIS, and then returning. But it is hardly unique in the scale of this problem.
• Continental Europe has a specific additional problem that is not as widely shared in the United Kingdom or the United States: Its criminal networks largely overlap with its extremist and/or terrorist networks. This point may be irrelevant to the Nice attack, but more widely, extremists in France or Belgium can make use of illicit channels for moving people, money, and weapons that are less available to would-be jihadis in places like the U.K. (where the criminal networks have more of a Caribbean and sub-Saharan African character, meaning they overlap less with extremist networks).
• Of course, the greatest numbers of terrorist attacks by Muslim extremists occur in the broader Muslim world, with Muslims as the primary victims—from Iraq and Syria to Libya and Yemen and Somalia to South Asia. French domestic policies have no bearing on these, of course.

There is no doubt that good work by counterterrorism and intelligence forces is crucial to preventing future attacks. France has done well in this regard—though it surely can do better, and it is surely trying to get better. There is also no doubt that promoting social cohesion in a broad sense is a worthy goal. But I would hesitate, personally, to attribute any apparent trend line in major attacks in the West to a particular policy of a country like France—especially when the latter is in fact doing much to seek to build bridges, as a matter of national policy, with Muslims at home and abroad. 

There is much more to do in promoting social cohesion, to be sure, even here in America (though our own problems probably center more on race than on religion at the moment). But the Nice attacker almost assuredly didn’t attack because his estranged wife couldn’t wear a veil in the manner and/or places she wanted. At a moment like this in particular, I disagree with insinuations to the contrary.

Media coverage of the Islamic State frequently refers to the group’s violent and seemingly archaic justice system without considering the institutional structures that enable this violence, or the broader function that it serves in the group’s ambitious state-building project. Legal institutions make it easier for the group to capture and retain territory by legitimizing its claim to sovereignty, justifying the expropriation of the property and land of enemies, and building goodwill with civilians by ensuring accountability.

The Islamic State’s legal system purports to strictly apply the divinely revealed body of Islamic law known as Sharia, which it regards as the only legitimate basis for governance. Although its legal system is frequently characterized as medieval, it has instrumentally supplemented the original text of the Quran with the modern rules and regulations that are needed to govern a 21st century state and punish modern day offenses—for example, traffic violations. It has the same three features that are present in any modern legal system: police, courts, and prisons.

In a region that has long been plagued by corruption, the Islamic State has attempted to ingratiate itself with civilians by claiming that its legal system is comparatively more legitimate and effective than the available alternatives. However, two emerging vulnerabilities—the system’s susceptibility to corruption and propensity for extra-legal violence—are increasingly undermining the Islamic State’s ability to obtain the trust and cooperation of civilians. Counterinsurgency efforts should be designed to undermine the legitimacy of its institutions. Long-term solutions in the region must involve a fundamental reorganization of political and legal institutions in ways that promote legitimacy and rule of law.

Downloads

• The legal foundations of the Islamic State

Event Information

On July 20 and 21, defense ministers from several nations will gather in Washington, D.C. at the invitation of U.S. Secretary of Defense Ash Carter. The meeting will bring together representatives from countries working to confront and defeat the Islamic State (or ISIL). French Defense Minister Jean-Yves Le Drian will be among those at the summit discussing how to accelerate long-term efforts to fight ISIL in Iraq and Syria. The close relationship between France and the United States has provided a solid base for security cooperation for decades, and in recent years, France has become one of America’s strongest allies in fighting terrorism and a prominent member of the international coalition to defeat ISIL.

On July 20, the Foreign Policy program at Brookings hosted Minister Le Drian for a discussion on French and U.S. cooperation as the two countries face multiple transnational security threats. Since becoming France’s defense minister in 2012, Le Drian has had to address numerous new security crises emerging from Africa, the Middle East, and within Europe itself. France faced horrific terrorist attacks on its own soil in January and November 2015 and remains under a state of emergency with its armed forces playing an active role in maintaining security both at home and abroad. Le Drian recently authored “Qui est l’ennemi?” (“Who is the enemy?”, Editions du Cerf, May 2016), defining a comprehensive strategy to address numerous current threats.

Join the conversation on Twitter using #USFrance

Video

• Introduction and featured speaker
• Discussion
• Introduction et conférencier invité
• Débat

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• 20160720_france_defense_transcript

France has been struck by an unprecedented three terror attacks in the last 18 months. In what’s called Operation Sentinelle, 13,000 French military personnel now patrol streets and protect key sites across the country, assisting police and other security agencies. “The fact that the armed forces are visible,” said French Defense Minister Jean-Yves Le Drian at a Brookings event on July 20, “help to reassure the French people that they are safe both at home and abroad.”

Do the challenges facing France today mean that it should reduce its engagement overseas, focusing instead on security at home? Le Drian doesn’t think so, and I agree. In a new book titled "Who is the Enemy?," he particularly emphasizes the multifaceted ISIS threat. As he said at Brookings: 

"Every war [has] two enemies…[Today’s] war [with ISIS] also sets in place two concepts of the “enemy” that are radically different: From a strategic point of view, we are dealing with a proto-state; at the heart of this entity, there is a terrorist army." 

It only further complicates matters, of course, that France faces ISIS threats on several fronts: in Syria and Iraq, on the one hand, and also on its own territory. This, Le Drian stressed, means “we must seek coherence in our military action.” It also helps explain why France remains one of the most active countries in the fight against the so-called Islamic State, as well as other extremist groups in the Middle East and in sub-Saharan Africa. In 2013 and 2014, France intervened in Mali in order to prevent jihadi groups from taking over the country. The French military also has a presence in Djibouti, Lebanon, Côte d’Ivoire, the Central African Republic, Gabon, Senegal, as well as in the Pacific (in French Polynesia and New Caledonia)—not to mention Syria, where France uses the Mediterranean Sea-based Charles de Gaulle aircraft carrier to strike ISIS targets. 

France is not a warmongering country—rather, it is responding to the fact that overseas threats come to it. Although it remains unclear to what extent the Nice attacker had connections with foreign terrorist networks, it has been established that the November 2015 Paris attacks were planned and orchestrated from Syria. This, among other considerations, has prompted France to engage further in Iraq and Syria. The rationale, as Minister Le Drian explained, is: 

“[T]rading our peace by reducing our military involvement doesn’t make sense. The more we let ISIS consolidate its presence on the Middle East, the more it will gather resources, attract fighters, and plan more attacks against us.” 

Team player

French policy isn’t just about ensuring its own security—rather, its many contributions are integrated within global efforts, including U.S.-led ones. As Le Drian said at Brookings: “I am convinced that the French-American relationship is stronger and better than ever.” France is a prominent participant of the 66-member international coalition against ISIS, and in that capacity participated in the first joint meeting of that group’s foreign and defense ministers in Washington this month. 

France remains a key member of the joint military operation Inherent Resolve in Iraq and Syria, which has damaged or destroyed over 26,0000 ISIS-related targets since August 2014. The Charles de Gaulle carrier—with 26 aircrafts on board—has been an essential part of that coalition mission. Following specific instructions from U.S. Defense Secretary Ash Carter and his counterpart in Paris, Minister Le Drian, French and U.S. intelligence agencies cooperate closely in intelligence-sharing. And just last week, President François Hollande announced that France will soon be supplying artillery to Iraq to support its fight against ISIS. Beyond the Iraq-Syria theater, France is cooperating with the United States and other partners in Libya, another country that is both a victim and source of extremist threats. 

The French Defense Ministry’s efforts to double-down on protecting French citizens within France, therefore, has not reduced its overseas role. Particularly now that the United Kingdom will leave the European Union, France’s military role has never been so important. France—along with Germany, which recently suggested it would raise its defense spending significantly—should continue to play a leading role as one the top defense actors in the West. 

Media coverage of the Islamic State has focused on the group’s grotesque use of violence and archaic governance style. Less attention has been paid, however, to the institutions that make those practices possible—institutions that lend the group legitimacy, at least in the eyes of supporters, as a sovereign state. In her new Brookings Analysis Paper, “The legal foundations of the Islamic State,” Mara Revkin argues that legal institutions play a critical role in the Islamic State’s state-building project. Those structures help the group take and keep territory, as well as provide a measure of accountability to the people living under its rule.

Lesser evil?

Revkin writes that “the Islamic State has attempted to ingratiate itself with civilians by claiming that its legal system is comparatively more legitimate and effective than the available alternatives.” The Syrian and Iraqi governments, Revkin explains, are often perceived as being highly corrupt and ineffective. The Islamic State is able to gain civilians’ favor by arguing that its political and legal institutions are more legitimate than those of the Syrian and Iraqi governments or rival armed groups. She adds: “some Syrians and Iraqis seem to prefer the legal system of the Islamic State to the available alternatives not because they agree with its ideology, but simply because they regard it as the lesser evil.” 

The Syrian and Iraqi governments...are often perceived as being highly corrupt and ineffective.

Revkin writes that for the Islamic State, shariah law is “the only legitimate basis for governance.” In cases where shariah fails to address modern-day problems, she explains, religiously legitimate authorities appointed by the Islamic State—such as military commanders, police officers, and the caliph himself—can issue legal decisions as long as they do not conflict with the divine rules of shariah or harm the welfare of the greater Muslim community. Alongside this is a system of rules and regulations to “govern civilians, discipline its own officials and combatants, and control territory” in areas of rights and duties, behavior, property, trade, and warfare. 

Making the state possible

Legal institutions help the Islamic State advance three main state-building objectives, in Revkin’s view: 

1. First, they support the Islamic State’s territorial expansion by “legitimizing [its] claims to sovereignty, justifying the expropriation of the property and land of enemies, and building goodwill with civilians.” 
2. Legal institutions also allow the Islamic State to enforce compliance and accountability of its own members and maintain internal control and discipline. Revkin describes various types of punishments the Islamic State uses to discipline its own members—these punishments are important, she writes, because “no government can establish itself as legitimate and sovereign without policing the behavior of the people who are responsible for implementing its policies.”
3. Finally, Revkin explores the legal institutions surrounding the Islamic State’s tax policies, which are “critical to financing the Islamic State’s governance and military operations.” Courts and judges, she explains, are crucial to “administering and legitimizing” taxation and justifying “economic activities that might otherwise resemble theft.” 

Weaknesses in the system

Although the Islamic State claims to have legitimate governing authority, based on a defined legal system, that system faces vulnerabilities. Revkin writes, for instance, that reports of corruption and extra-legal violence are “threatening the organization’s long-term sustainability and undermining its ability to win the trust and cooperation of civilians.”

Amid recent signs that the group is losing strength, Revkin argues that it’s struggling to maintain its own moral standards. To further weaken the Islamic State, she recommends working to undermine those institutions. The trouble is, as Revkin points out: “the Islamic State came to power largely by exploiting the weakness and illegitimacy of existing institutions” in Iraq and Syria. Thus, a sustainable plan for ultimately destroying the organization must also involve strengthening political and legal institutions in those countries. 

Since mid-2014, the world’s attention has been transfixed on the aesthetically shocking actions of ISIS and the threat it poses to regional and international security. However, it is arguably Jabhat al-Nusra in Syria—and perhaps the al-Qaida movement more broadly—that looks more likely to survive over the long term and to threaten local, regional and international security interests. Since its emergence in Syria in late 2011, Jabhat al-Nusra has transformed itself from an unpopular outsider accused of Islamic State in Iraq (ISI)-like brutality towards one of the most powerful armed actors in the Syrian crisis. Moreover, its break away from the ISI in April 2013 set it further down a path of deep integration into the broader Syrian armed opposition in its fight against Bashar Assad’s regime.

Nearly five years after its formation, Jabhat al-Nusra has demonstrated the potential value of its "long game" approach. By adopting a strategy of gradualism, it has socialized populations into first accepting, and then supporting and defending, this al-Qaida-like movement. Jabhat al-Nusra aims to epitomize the realization of al-Qaida’s evolved thinking. It seeks to build localized bases of influence by embedding itself in within popular revolutionary dynamics and, eventually, establish zones of territorial control from which it can launch attacks against the Western world. By establishing a durable presence in Syria and potentially considering separating itself from al-Qaida internationally, Jabhat al-Nusra seeks to realize its long-term vision of establishing Islamic Emirates inside Syria, as components of a future Caliphate.

Jabhat al-Nusra has successfully prepared its surroundings in such a way as to give it an improved chance of surviving in the long-term, despite international counterterrorism efforts. The international community must work to de-escalate the situation in Syria and more forcefully push for a diplomatic settlement so as to prevent the establishment of a longstanding jihadi safe haven. Policy decisions, including expanding support to Syria’s mainstream opposition, encouraging opposition-Kurdish dialogue, and interrupting al-Qaida’s finances, can diminish Jabhat al-Nusra’s chances of survival.

Downloads

• Profiling Jabhat al-Nusra

Introduction

The global community has set itself the challenge of meeting the Millennium Development Goals (MDGs) by 2015 as a way to combat world poverty and hunger. In 2007, the halfway point, it is clear that many countries will not be able to meet the MDGs without undertaking significantly greater efforts. One constraint that needs to be overcome is that development interventions—projects, programs, policies—are all too often like small pebbles thrown into a big pond: they are limited in scale, short-lived, and therefore have little lasting impact. This may explain why so many studies have found that external aid has had weak or no development impact in the aggregate, even though many individual interventions have been successful in terms of their project- or program-specific goals.

Confronted with the challenge of meeting the MDGs, the development community has recently begun to focus on the need to scale up interventions. Scaling up means taking successful projects, programs, or policies and expanding, adapting, and sustaining them in different ways over time for greater development impact. This emphasis on scaling up has emerged from concern over how to deploy and absorb the substantially increased levels of official development assistance that were promised by the wealthy countries at recent G8 summits. A fragmented aid architecture complicates this task; multilateral, bilateral, and private aid entities have multiplied, leading to many more—but smaller— aid projects and programs and increasing transaction costs for recipient countries. In response, some aid donors have started to move from project to program support, and in the Paris Declaration, official donors committed themselves to work together for better coordinated aid delivery.

The current focus on scaling up is not entirely new, however. During the 1980s, as nongovernmental organizations (NGOs) increasingly began to engage in development activities, scaling up emerged as a challenge. NGO interventions were (and are) typically small in scale and often apply new approaches. Therefore, the question of how to replicate and scale up successful models gained prominence even then, especially in connection with participatory and community development approaches. Indeed, the current interest among philanthropic foundations and NGOs in how to scale up their interventions is an echo of these earlier concerns.

In response to this increased focus on scaling up—and its increased urgency—this policy brief takes a comprehensive look at what the literature and experience have to say about whether and how to scale up development interventions.

Downloads

• Download

Abstract

Scaling up of development interventions is much debated today as a way to improve their impact and effectiveness. Based on a review of scaling up literature and practice, this paper develops a framework for the key dynamics that allow the scaling up process to happen. The authors explore the possible approaches and paths to scaling up, the drivers of expansion and of replication, the space that has to be created for interventions to grow, and the role of evaluation and of careful planning and implementation. They draw a number of lessons for the development analyst and practitioner. More than anything else, scaling up is about political and organizational leadership, about vision, values and mindset, and about incentives and accountability—all oriented to make scaling up a central element of individual, institutional, national and international development efforts. The paper concludes by highlighting some implications for aid and aid donors.

An annotated bibliography of the literature on scaling up and development aid effectiveness was created by Oksana Pidufala to supplement this working paper. Read more »

Downloads

• Download

Editor's Note: The World Bank’s Independent Evaluation Group (IEG) recently published a self-evaluation of its activities. Besides representing current thinking among evaluation experts at the World Bank, it also more broadly reflects some of the strengths and gaps in the approaches that evaluators use to assess and learn from the performance of the international institutions with which they work. The old question “Quis custodet ipsos custodes?” – loosely translated as “Who evaluates the evaluators?” – remains as relevant as ever. Johannes Linn served as an external peer reviewer of the self-evaluation and provides a bird’s-eye view on the lessons learned.

An Overview of the World Bank’s IEG Self-Evaluation Report

In 2011 the World Bank’s Independent Evaluation Group (IEG) carried out and published a self-evaluation of its activities. The self-evaluation team was led by an internal manager, but involved a respected external evaluation expert as the principal author and also an external peer reviewer.

The IEG self-evaluation follows best professional practices as codified by the Evaluation Cooperation Group (ECG). This group brings together the evaluation offices of seven major multilateral financial institutions in joint efforts designed to enhance evaluation performance and cooperation among their evaluators. One can therefore infer that the approach and focus of the IEG self-evaluation is representative of a broader set of practices that are currently used by the evaluation community of international financial organizations.

At the outset the IEG report states that “IEG is the largest evaluation department among Evaluation Capacity Group (ECG) members and is held in high regard by the international evaluation community. Independent assessments of IEG’s role as an independent evaluation function for the Bank and IFC rated it above the evaluation functions in most other ECG members, international nongovernmental organizations, and transnational corporations and found that IEG follows good practice evaluation principles.”

The self-evaluation report generally confirms this positive assessment. For four out of six areas of its mandate IEG gives itself the second highest rating (“good”) out of six possible rating categories. This includes (a) the professional quality of its evaluations, (b) its reports on how the World Bank’s management follows up on IEG recommendations, (c) cooperation with other evaluation offices, and (d) assistance to borrowing countries in improving their own evaluation capacity. In the area of appraising the World Bank’s self-evaluation and risk management practices, the report offers the third highest rating (“satisfactory”), while it gives the third lowest rating (“modest”) for IEG’s impact on the Bank’s policies, strategies and operations. In addition the self-evaluation concludes that overall the performance of IEG has been “good” and that it operates independently, effectively and efficiently.

The report makes a number of recommendations for improvement, which are likely to be helpful, but have limited impact on its activities. They cover measures to further enhance the independence of IEG and the consistency of evaluation practices as applied across the World Bank Group’s branches – the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) –; to improve the design of evaluations and the engagement with Bank management upstream for greater impact; and monitoring the impact of recent organizational changes in IEG in terms of results achieved. The report also recommends that more be done to evaluate the Bank’s analytical work and that evaluations draw on comparative evidence.

Assessment

In terms of the parameters of self-evaluation set by the prevailing practice among the evaluators on international financial agencies, the IEG self-evaluation is accurate and helpful. From my own experience as an operational manager in the Bank whose activities were evaluated by IEG in years past, and as a user of IEG evaluations (and of evaluations of other international aid organizations) for my research on aid effectiveness, I concur that IEG is independent and effective in meeting its mandate as defined. Moreover, the self-evaluation produces useful quantitative evidence (including survey results, budget analysis, etc.) to corroborate qualitative judgments.

However, the self-evaluation suffers from a number of limitations in approach and gaps in focus, which are broadly representative of the practices prevalent among many of the evaluation offices of international aid agencies.

Approach of the IEG self-evaluation

The core of the self-evaluation report is about the evaluation process followed by IEG, with very little said about the substance of IEG’s evaluations. The following questions could have usefully been raised, but were not: do evaluations cover the right issues with the right intensity, such as growth and poverty; environmental, governance, and gender impacts; regional dimensions versus exclusive country or project focus; effectiveness in addressing the problems of fragile and conflict states; effectiveness in dealing with global public goods; sustainability and scaling up; etc. Therefore the report does not deal with the question of whether IEG effectively responds in its evaluations to the many important strategic debates and issues with which the development community is grappling.

Related to this limitation is the fact that the report assessed the quality of IEG’s mostly in terms of (a) whether its approach and processes meet certain standards established by the Evaluation Cooperation Group; and (b) how it is judged by stakeholders in response to a survey commissioned for this evaluation. Both these approaches are useful, but they do not have any basis in professional assessments of the quality of individual products. This is equivalent to IEG evaluating the World Bank’s projects on the quality of its processes (e.g., appraisal and supervision processes) and on the basis of stakeholder surveys, without evaluating individual products and their impacts.

Gaps in the Self-Evaluation and in Evaluation Practice

Careful reading of the report reveals six important gaps in the IEG self-evaluation, in the prevailing evaluation practice in the World Bank, and more generally in the way international financial organizations evaluate their own performance. The first three gaps relate to aspects of the evaluation approach used and the second three gaps relate to lack of focus in the self-evaluation on key internal organizational issues:

1. Impact Evaluations: The report notes that IEG carries out two to three impact evaluations per year, but it sidesteps the debate in the current evaluation literature and practice as to what extent the “gold standard” of randomized impact evaluation should occupy a much more central role. Given the importance of this debate and divergence of views, it would have been appropriate for the self-evaluation to assess IEG’s current practice of very limited use of randomized evaluations.

2. Evaluation of Scaling Up: The report does not address the question of to what extent current IEG practice not only assesses the performance of individual projects in terms of their outcomes and sustainability, but also in terms of whether the Bank has systematically built on its experience in specific projects to help scale up their impact through support for expansion or replication in follow-up operations or through effective hand-off to the government or other partners. In fact, currently IEG does not explicitly and systematically consider scaling up in its project and program evaluations. For example, in a recent IEG evaluation of World Bank funded municipal development projects (MDPs) , IEG found that the Bank has supported multiple MDPs in many countries over the years, but the evaluation did not address the obvious question whether the Bank systematically planned for the project sequence or built on its experience from prior projects in subsequent operations. While most other evaluation offices like IEG do not consider scaling up, some (in particular those of the International Fund for Agricultural Development and the United Nations Development Program) have started doing so in recent years.

3. Drawing on the Experience of and Benchmarking Against Other Institutions: The self-evaluation report does a good job in benchmarking IEG performance in a number of respects against that of other multilateral institutions. In the main text of the report it states that “IEG plans to develop guidelines for approach papers to ensure greater quality, in particular in drawing on comparative information from other sources and benchmarking against other institutions.” This is a welcome intention, but it is inadequately motivated in the rest of the report and not reflected in the Executive Summary. The reality is that IEG, like most multilateral evaluation offices, so far has not systematically drawn on the evaluations and relevant experience of other aid agencies in its evaluations of World Bank performance. This has severely limited the learning impact of the evaluations.

4. Bank Internal Policies, Management Processes and Incentives: IEG evaluations traditionally do not focus on how the Bank’s internal policies, management and incentives affect the quality of Bank engagement in countries. Therefore evaluations cannot offer any insights into whether and how Bank-internal operating modalities contribute to results. Two recent exceptions are notable exceptions. First, the IEG evaluation of the Bank’s approach to harmonization with other donors and alignment with country priorities assesses the incentives for staff to support harmonization and alignment. The evaluation concludes that there are insufficient incentives, a finding disputed by management. Second, is the evaluation of the Bank’s internal matrix management arrangements, which is currently under way. The self-evaluation notes that Bank management tried to quash the matrix evaluation on the grounds that it did not fall under the mandate of IEG. This is an unfortunate argument, since an assessment of the institutional reasons for the Bank’s performance is an essential component of any meaningful evaluation of Bank-supported programs. While making a good case for the specific instance of the matrix evaluation, the self-evaluation report shies away from a more general statement in support of engaging IEG on issues of Bank-internal policies, management processes and incentives. It is notable that IFAD’s Independent Office of Evaluation appears to be more aggressive in this regard: It currently is carrying out a full evaluation of IFAD’s internal efficiency and previous evaluations (e.g., an evaluation of innovation and scaling up) did not shy away from assessing internal institutional dimensions.

5. World Bank Governance: The IEG self-evaluation is even more restrictive in how it interprets its mandate regarding the evaluation of the World Bank’s governance structures and processes (including its approach to members’ voice and vote, the functioning of its board of directors, the selection of its senior management, etc.). It considers these topics beyond IEG’s mandate. This is unfortunate, since the way the Bank’s governance evolves will substantially affect its long-term legitimacy, effectiveness and viability as an international financial institution. Since IEG reports to the Bank’s board of directors, and many of the governance issues involve questions of the board’s composition, role and functioning, there is a valid question of how effectively IEG could carry out such an evaluation. However, it is notable that the IMF’s Independent Evaluation Office, which similarly reports to the IMF board of directors, published a full evaluation of the IMF’s governance in 2008, which effectively addressed many of the right questions.

6. Synergies between World Bank, IFC and MIGA: The self-evaluation report points out that the recent internal reorganization of IEG aimed to assure more effective and consistent evaluations across the three member branches of the World Bank Group. This is welcome, but the report does not assess how past evaluations addressed the question of whether the World Bank, IFC and MIGA effectively capitalized on the potential synergies among the three organizations. The recent evaluation of the World Bank Group’s response to the global economic crisis of 2008/9 provided parallel assessments of each agency’s performance, but did not address whether they work together effectively in maximizing their synergies. The reality is that the three organizations have deeply engrained institutional cultures and generally go their own ways rather than closely coordinating their activities on the ground. Future evaluations should explicitly consider whether the three effectively cooperate or not. While the World Bank is unique in the way it has organizationally separated its private sector and guarantee operations, other aid organizations also have problems of a lack of cooperation, coordination and synergy among different units within the agency. Therefore, the same comment also applies to their evaluation approaches.

Conclusions

Self-evaluations are valuable tools for performance assessment and IEG is to be congratulated for carrying out and publishing such an evaluation of its own activities. As for all self-evaluations, it should be seen as an input to an independent external evaluation, a decision that, for now, has apparently been postponed by the Bank’s board of directors.

IEG’s self-evaluation has many strengths and provides an overall positive assessment of IEG’s work. However, it does reflect some important limitations of analysis and of certain gaps in approach and coverage, which an independent external review should consider explicitly, and which IEG’s management should address. Since many of these issues also likely apply to most of the other evaluation approaches by other evaluation offices, the lessons have relevance beyond IEG and the World Bank.

Key lessons include:

• An evaluation of evaluations should focus not only on process, but also on the substantive issues that the institution is grappling with.
• An evaluation of the effectiveness of evaluations should include a professional assessment of the quality of evaluation products.
• An evaluation of evaluations should assess:
  o How effectively impact evaluations are used;
  o How scaling up of successful interventions is treated;
  o How the experience of other comparable institutions is utilized;
  o Whether and how the internal policies, management practices and incentives of the institution are effectively assessed;
  o Whether and how the governance of the institution is evaluated; and
  o Whether and how internal coordination, cooperation and synergy among units within the organizations are assessed.

Evaluations play an essential role in the accountability and learning of international aid organizations. Hence it is critical that evaluations address the right issues and use appropriate techniques. If the lessons above were reflected in the evaluation practices of the aid institutions, this would represent a significant step forward in the quality, relevance and likely impact of evaluations.

Today, Egyptian President Abdel-Fattah el-Sissi is witnessing the most vocal and angry objection to his rule since he took power via a military coup in 2013. Across Cairo and beyond, Egyptians are gathering and chanting some of the same slogans from the January 2011 revolution—such as “the people want the fall of the regime” and “down with military rule.” These protests are not a spontaneous uprising. They were planned and announced on April 15, when thousands of Egyptians took to the streets, protesting the latest in a series of bold and controversial decisions that are slowly and steadily chipping away at Sissi’s once solid support structure abroad and at home.

During Saudi King Salman’s recent visit to Cairo, the Egyptian government announced that it had agreed to transfer sovereignty of two Red Sea islands—Tiran and Sanafir—to Saudi Arabia. This decision, which coincided with a $22 billion oil and aid deal, has a clear short term pay-off: a substantial Band-Aid on Egypt’s gaping economic wounds. But Sissi and his government are once again dramatically underestimating just how self-destructive their behavior can be. As my colleague Tamara Wittes eloquently noted, Egypt “continues to throw obstacles in the road of U.S.-Egyptian cooperation.” But even worse than the self-sabotage in Egypt’s foreign relations is the damage Sissi is doing to his reputation at home. 

The decision to transfer the islands to Saudi Arabia may turn out to be the final nail in Sissi’s coffin.

To the streets, again

Following the announcement of this decision, Egyptians took to Twitter, with the hashtag “leave” and “I didn’t elect Sissi” trending in Egypt. Lawyers filed lawsuits in Egyptian courts opposing the agreement. And plans were made for a much larger protest today, Sinai Liberation Day. 

But today’s protests are different than in the past. First, while the anti-Sissi protesters had time to plan and coordinate their actions, so did the regime. Today, pro-Sissi supporters organized their own protests, proudly waving the Saudi flag in Cairo’s symbolic Tahrir Square. The Egyptian Air Force painted the Egyptian flag in the sky. And the security forces came out in droves early today across greater Cairo, closing off access to most of the usual protests sites (such as the Journalists’ Syndicate and the Doctors’ Syndicate) and making a massive show of force to deter people from coming out. 

The government clearly learned a few lessons since Mubarak’s fall. A law passed in 2013 requires pre-approval from the Interior Ministry for any protest activity. That gave Sissi’s henchmen a green light to round up actual and suspected protesters as they have been doing since Thursday, arresting hundreds of suspected agitators and human rights activists on charges related to organizing today’s protests. (Notably, the pro-Sissi demonstrators have not been touched.) As each new anti-regime protest pops up today, security forces are there, arresting protesters and journalists and dispersing them with tear gas and rubber bullets. Regardless of the final outcome of today’s events, Sissi should pay attention to the growing dissatisfaction among the Egyptian people. 

The symbolism of holding today’s protests on Sinai Liberation Day is potent. Threats to Egypt’s nationalism and national sovereignty have long been key drivers of Egyptian rage, allowing the protest organizers to tap in to the anger and frustration shared by Egyptians across the political spectrum. The outrage citizens have expressed in the streets, online, and in the media should be a red flag to Sissi, who is hemorrhaging support. 

Notably, he’s now struck a nerve not just with Islamists or others in the anti-Sissi crowd, but with one of the few remaining bastions of Sissi supporters—the everyday Egyptians who are not normally politically engaged. This is a group of people who, following five years of political turmoil, see Sissi as Egypt’s best chance at stability in an increasingly unstable neighborhood. And they’re generally willing to forgive Sissi for his transgressions. They don’t believe the theory that the Egyptian security services are responsible for Italian PhD student Giulio Regeni’s death. They agree that foreign funding of NGOs is a form of Western meddling in Egyptian affairs. They justify the brutal crackdown on free expression in the name of security. But secretly concocting a deal to give away Egyptian land—that is one pill even they can’t swallow. 

Final straws?

Making matters worse are reports that Egypt consulted with Israel and the United States prior to the transfer. While the Israeli-Egyptian peace treaty remains active, Egypt and Israel’s peace is cold, at best. The notion that Sissi would consult with Israel over something that he kept secret from his own people is the ultimate insult and betrayal to many Egyptians. The facts behind the transfer matter very little. What matters is the perception of the Egyptian public that President Sissi has duped them. 

The decision to transfer the islands to Saudi Arabia may turn out to be the final nail in Sissi’s coffin. Over the past several months, he has lost other pillars of support—including secular revolutionaries, who saw former President Morsi and the Muslim Brotherhood as subverting the revolution and supported the military’s return to power. The far-reaching and brutal crackdown on Egyptian journalists and NGOs turned many of them off from Sissi. And wealthy Egyptians, who believed Sissi’s promises to grow the economy and protect their assets, have increasingly questioned their leader as Egypt’s economy continues to plummet. 

Sissi is not only running out of supporters, he is also running out of excuses.

Sissi is not only running out of supporters, he is also running out of excuses. Rather than admit his mistakes, Sissi has defended his actions, shifting the blame and feeding conspiracy theories. While protests were growing across Egypt on April 15, Sissi spoke to a group of Egyptian youth, referencing a “hellish scheme” to destabilize Egypt from within. 

Unfortunately for Sissi, there is no such “scheme.” In 2011 it was not a Western plot, as some Egyptian conspiracy theories have suggested, that ousted Mubarak—it was the Egyptian people, fed up with actions Mubarak carried out as president. In 2013, the coup that ousted Morsi succeeded because the people were fed up with decisions he made in office to consolidate power and reject democratic reforms. Had either Mubarak or Morsi spent as much time responding to the wants and needs of their citizenry as they had quashing dissent, one of them might still be in office. Much like his predecessors, what Sissi fails to understand is that the thing most likely to destabilize his government is neither an external conspiracy not an internal scheme—it’s him. 

Editors’ Note: Donald Trump has exposed the tension between democracy and liberal values—similar to the Arab Spring, writes Shadi Hamid. This piece originally appeared on The Atlantic.

When I was living in the Middle East, politics always felt existential, in a way that I suppose I could never fully understand. After all, I could always leave (as my relatives in Egypt were fond of reminding me). But it was easy enough to sense it. Here, in the era of Arab revolt, elections really had consequences. Politics wasn’t about policy; it was about a battle over the very meaning and purpose of the nation-state. These were the things that mattered more than anything else, in part because they were impossible to measure or quantify.

The primary divide in most Arab countries was between Islamists and non-Islamists. The latter, especially those of a more secular bent, feared that Islamist rule, however “democratic” it might be, would alter the nature of their countries beyond recognition. It wouldn’t just affect their governments or their laws, but how they lived, what they wore, and how they raised their sons and daughters.

Perhaps more than at any other time, millions of Americans are getting a sense, however mild in comparison, of what it might feel like to lose your country—or at least think about losing your country—because of what people decide to do in the privacy of the voting booth. It still remains (somewhat) unlikely that Donald Trump, the now presumptive Republican nominee, can win a general election. Regardless of the final outcome, however, the billionaire’s rise offers up a powerful—and frightening—reminder that liberal democracy, even where it’s most entrenched, is a fragile thing.

* * *

When I hear my friends debating how, exactly, so many of their fellow citizens could support someone like Trump, it reminds me a bit of Egypt. In my forthcoming book, I relay a telling conversation I had four years ago, which has stayed with me since. A few days after the country’s first post-revolutionary elections concluded in January 2012, I visited my great aunt in her extravagant flat in the posh Cairo suburb of Heliopolis. She was in a state of shock, but worse than that was the confusion. It was one thing for the Muslim Brotherhood, long Egypt’s largest opposition group, to win close to 40 percent of the vote, but how could 28 percent of Egyptians vote for ultraconservative Salafi parties, which believed in the strict implementation of Islamic law?

Like most Egyptians, she personally knew Brotherhood members even if she didn’t quite like them, but she hadn’t had much experience with Salafis and seemed totally unaware that they had extended their reach deep into Egyptian society. She realized, perhaps for the first time, that the country she had thought was hers for the better part of 70 years would never quite be the same. It hadn’t really even been hers to begin with.

What if voters don’t want to be liberal and vote accordingly?

What my aunt feared was that Egypt would become an “illiberal democracy,” a term popularized by Fareed Zakaria in his 2003 book The Future of Freedom, but one that’s still difficult for Americans to fundamentally relate to. In the American experience, democracy and liberalism seemed to go hand in hand, to such an extent that democracy really just became shorthand for “liberal democracy.”

As Richard Youngs writes in his excellent study of non-Western democracy, liberalism and democracy have historically been “rival notions and not bedfellows.” Liberalism is about non-negotiable personal rights and freedoms. Democracy, while requiring some basic protection of rights to allow for meaningful competition, is more about popular sovereignty, popular will, and accountability and responsiveness to the voting public. Which, of course, raises the question: What if voters don’t want to be liberal and vote accordingly?

* * *

When the stakes are high, there is more to lose, and if there is more to lose, those on the losing end of a ballot box have powerful incentives to play “spoiler.” Fortunately, in the post-Civil War United States, the stakes have never reached what political scientist Barry Weingast calls the “threshold” at which citizens decide to defend themselves through extra-constitutional means, including by appealing for the military to take sides. This, in part, is why (good) constitutions are so important: They lower the stakes, reassuring citizens that even if their preferred party loses the election, it’s still just that—an election.

Donald Trump, or more specifically what he represents, calls some of these assumptions into question. Trump himself isn’t quite an Islamist, but he is a proponent of a kind of “illiberal democracy,” even if he himself may not be familiar with the term. Drawing on a wellspring of white nativism and machismo, candidate Trump has regularly made demeaning statements about entire groups of people, including African-Americans, Mexicans, and women. His commitment to the protections enshrined in U.S. constitution are questionable, at best, and if we assume the worst, downright frightening (the difficulty with Trump is that he’s not precise with words, so it’s sometimes hard to make sense of what he’s saying). He has expressed support for registering Muslims in a database, elaborating that they could “sign up at different places.” When a reporter asked how this was different from requiring Jews to register in Nazi Germany, Trump said “you tell me,” prompting The Atlantic’s David Graham to note that “it’s hard to remember a time when a supposedly mainstream candidate had no interest in differentiating ideas he’s endorsed from those of the Nazis.” Trump, for good measure, has also refused to disavow President Franklin D. Roosevelt’s internment of Japanese-Americans.

The U.S. Constitution includes robust civil-liberties protections, enshrined in the Bill of Rights. But these protections are not unlimited. Contrary to popular belief, majorities—if they’re large enough—can, in fact, do nearly anything they want, even in established democracies. It’s only really a question of how high the majoritarian bar is. In the United States, two-thirds of Congress and 75 percent of the states can amend or repeal articles of the Constitution. They could theoretically pass a constitutional amendment banning abortion. In countries like Egypt, Tunisia, and Turkey, where alcohol is currently legal and relatively easy to find, the issue of alcohol consumption is a touchstone for endless “what if” hypothesizing. Yet, Prohibition happened not in any of those countries but in America, with large majorities in the Senate and House of Representatives as well as 46 of 48 states backing the 18th Amendment (of course, banning alcohol in the U.S. wasn’t justified on primarily scriptural grounds, while in Muslim-majority countries, prohibition is seen as fulfilling an explicitly Quranic directive).

In other words, built-in constraints and constitutional “guarantees” aren’t enough on their own to preclude illiberal outcomes. What Americans really depend on, then, is a shared political culture and the ideas and ideals that undergird it. As James Fallows notes, “Liberal democracies like ours depend on rules but also on norms—on the assumption that you’ll go so far, but no further, to advance your political ends.” But all it apparently takes is one man with charisma and an unusually perceptive understanding of the human psyche to change that. There are norms against politicians suggesting that minorities should have special identification cards. There are norms against saying you want to kill the families of terrorists. There are norms against encouraging your supporters to use violence against their political opponents. It’s not entirely clear why you don’t do or say these things (because Trump clearly has), but you just don’t. The very fact that Trump has made such frightening comments on national television—without any corresponding “disqualification” or decline in popular support—has already undermined these longstanding norms.

The United States has had demagogues before, but they rarely make for viable presidential candidates. This is democracy’s blessing as well as its curse: that people you really don’t like—people who you think might threaten the Republic—can actually win. In the specific context of the Republican nomination, Trump opponents basically called for prioritizing good outcomes over democratic ones. They continued to search for possible paths to denying Trump the nomination, despite the fact that, barring acts of God, he was certain to win the popular vote and a plurality of delegates in the primaries.

Even if Trump reached the magic number of 1,237 delegates, which would normally settle the matter, there were those who still seemed intent on scouring the rulebooks, parliamentary procedure, and delegate details in the hope of averting disaster. Democratic norms, the thinking goes, are great in normal contexts, but sometimes the stakes are simply too high to let democratic outcomes stand. As the columnist Walter Shapiro wrote, “[W]ith the threat of the first takeover of a modern political party by an authoritarian who traffics in racism and exudes contempt for the First Amendment ... [t]here would be nothing anti-democratic about GOP leaders using every mechanism in their power to stop Trump.” Nate Silver pointed out that “technically [Republicans would] be able to deny Trump the nomination even if he had a delegate majority by changing the rules at the last minute.” They could still theoretically do something like this, even after Trump’s decisive victory in Indiana. The Republican Party is not a country, and the party can disregard the preferences of primary voters if it so chooses, but elite pacts and back-room negotiations would seem decidedly antiquated during an unusually populist moment in American politics.

[T]here will no doubt be a temptation to defy or otherwise undermine a democratically elected Trump.

This particular debate in some ways mirrors arguments over the tensions between democracy and liberalism, a debate that will only intensify if Trump gains ground on Hillary Clinton in the coming months. It is probably time to err on the side of imagination, since party elites and pundits failed to imagine the unthinkable once already. What if Trump actually wins the presidency? How would we as Americans deal with an outcome that at least some of us see as a potential danger to our Constitution as well as our livelihoods?

If Donald Trump wins, he would have, whether we liked it or not, a democratic mandate. Once in power, he might moderate his rhetoric and policies (yet another data point in the debate over the “inclusion-moderation hypothesis”), rendering at least some of this discussion moot. Yet it’s also possible that, facing a growing terrorist threat and a sputtering economy, more and more Americans might, like their newly elected president, dispense with the norms of reasonable conduct and support extreme measures. Still, a President Trump would be a legitimate president, having been freely and fairly elected by enough Americans. He would be, as much as it pains me to say it, our president. Still, there will no doubt be a temptation to defy or otherwise undermine a democratically elected Trump. For those of us who study the Middle East, the idea of not respecting democratic outcomes is business as usual, but I never thought it would be up for debate in the United States.

* * *

“Deep state” is a phrase that’s used to describe the constellation of autonomous and self-perpetuating institutions, namely the judiciary, military, and security services, which operate outside the glare of the public and are immune to the electorate’s whims. This deep state, acting as the guardian of national identity, puts limits on what elected politicians can hope to accomplish. The deep state was responsible for four coups in Turkey, the most recent of which deposed the country’s first-ever democratically elected Islamist prime minister in 1997.

It would be difficult for Americans to think about their own government—or “regime”—in such terms. The U.S. military is subject to civilian control, while Supreme Court justices, though unelected and appointed to life terms, are nominated by the president and confirmed by the Senate. It is possible, however, to imagine a president so reckless as to activate state institutions against him or her, in a way that makes the notion of an American deep state more meaningful and relevant.

Former CIA Director Michael Hayden ignited some speculative debate when he said that the military “would refuse to act” if ordered by a President Trump to take actions that were clearly illegal, such as killing the families of terrorists. Moreover, he said, military commanders are “required not to follow an unlawful order.” Even short of flagrant illegality, the military can still do what it’s done, at times, with nearly every sitting president. Peter Feaver, a leading expert on civil-military relations, notes that “the historical record is replete with cases of the military shirking—withholding information and options, slow-rolling, end-runs to Congress and the media, inflating cost estimates, etc.—to thwart civilian policies they deem to be unwise.” Considering, however, that Trump would likely be more “unwise” than most past presidents, such tensions could intensify well beyond what America’s political system is accustomed to.

"[C]oup”...is not a word that Americans should ever get used to hearing in everyday political discourse.

One can also easily imagine left-of-center (and right-of-center) civil servants in the Departments of State and Defense working against the president from within to mitigate his effectiveness and even his authority. This would be good, insofar as Americans wouldn’t want their president doing things that were crazy, illegal, or both. But it would still raise difficult questions about democratic legitimacy and how far an elected president can pursue his preferred policies, especially when it comes to issues that aren’t clear-cut. If the military refused to obey orders, however justified their refusal, then it could very well erode norms against military intervention in domestic politics. In response to Hayden’s comments, host Bill Maher joked that the former CIA director was floating “a coup.” This is not a word that Americans should ever get used to hearing in everyday political discourse. The norm against “coups” is a powerful one, which explains why American analysts (if not the U.S. government) are generally uncomfortable with military coups in foreign countries. No one teaches us that military coups are bad. Rather, it’s something we absorb in the process of being American. It goes without saying, so it’s rarely said.

Recently, a few friends (who work on Middle East issues) and I had an interesting although ultimately frightening conversation, as Trump extended his delegate lead over Ted Cruz. Sometimes it’s useful to game out worst-case scenarios, however unlikely they might seem. We tried imagining a dystopian future and came up with internment camps, (threats of) military coups, and pro-Trump militias. Soon enough, the last didn’t seem nearly so farfetched, with volunteers offering to provide security at Trump rallies (for Trump supporters).

* * *

It is hard to imagine such things because, despite a long, low-intensity war on terrorism, America hasn’t faced a large-scale terrorist attack on the homeland since September 11, 2001. Democratic systems produce self-perpetuating norms, because they are accountable to a voting public. It’s this very responsiveness, though, that can be a source of vulnerability, if enough citizens, in the grip of fear, decide to prioritize “security” over liberty. As the legal scholar Christopher Kutz writes in the suggestively titled article “How Norms Die,” democracy can be “at the same time both fertile and toxic: fertile as a source of humanitarian values and institutions, but toxic to the very institutions it cultivates.”

This is something we can measure. As Daniel Bush observed, after analyzing Pew survey data from 2002 to 2014: “During each campaign season, respondents reported having a higher negative impression of Muslim Americans than in non-election years.” This is a bit more mild than the link between elections and religious riots in India. As the historian of religions Michael Cook notes, “There is no doubt that Hindu nationalist politicians believe that communal riots can get out the Hindu vote for them. ... Under the right conditions the communal riot is a winning [electoral] strategy.”

Norm shifting of an even more dangerous kind than India’s can happen rather quickly in countries where democracy is not yet consolidated. For example, millions of Egyptians who demanded freedom and democracy in 2011 turned seemingly against it in less than two and half years, supporting not just a return to authoritarian rule but the August 14, 2013 massacre of more than 800 protesters—what Human Rights Watch calls the “worst mass killing in [Egypt’s] modern history.”

The kinds of shifts that occur in established democracies are less nefarious, but they can happen just the same. Torture is a good example. Kutz calls the spread of global norms against torture “one of the most impressive successes of the post-war period.” Yet, in the United States, these norms began to erode after the attacks of September 11th. Soon enough, torture—or what some were now euphemistically calling “enhanced interrogation”—came to enjoy broad support among the American public. The lesson again is clear. However strong they may first appear, norms, particularly those relating to national security, are more fragile than we might like to think. Once their sanctity is undermined by authority figures (whether presidents or presidential candidates), others can judge that what was once considered shameful is now not just socially tolerated but also necessary, good, and just. This is why “political correctness”—even if it seems irritating and is sometimes abused to restrict reasonable debate—still represents a public good: It makes us think twice about saying things that might contribute to the erosion of liberal and democratic norms.

[N]orms, particularly those relating to national security, are more fragile than we might like to think.

We have now reached a point where current or former presidential candidates from both parties have flirted with the idea of internment camps (former Democratic candidate Wesley Clark has called for “segregating” radicalized Muslims who are “disloyal to the United States”). In a series of incidents that have received less attention, a Tennessee State Representative called for using state institutions, in this case the National Guard, to “round up” Syrian refugees. Meanwhile, the mayor of Roanoke, Virginia, called for suspending assistance to refugees, but went further in an official statement on government letterhead. “I’m reminded,” he wrote, “that President Franklin D. Roosevelt felt compelled to sequester Japanese foreign nationals after the bombing of Pearl Harbor, and it appears that the threat of harm to America from ISIS now is just as real and serious as that from our enemies then.”

No less than Supreme Court justice Antonin Scalia believed that it could happen here. On this, he is on strong ground, since it has, of course, already happened. In 1944, the Supreme Court upheld Roosevelt’s internment of Japanese-Americans in Korematsu v. United States. While Scalia said that the decision was “wrong,” he also issued a warning in his blunt style: “You are kidding yourself if you think the same thing will not happen again.”

The norm against internment has been undermined, even though Americans do not face anything close to the threat presented by the Nazis and Japan during World War II. Which raises the question of what a plurality, or even a majority, of Americans might be willing to support if they had to confront a threat that was truly existential. We Americans are not, today, at war, at least not in the normal sense. I hope to God that we never will be again. But we might be. And this is where Scalia’s words that day were perhaps most chilling, in part because he was right. Evoking the Latin expression inter arma enim silent leges, he reminded the audience that “in times of war, the laws fall silent.” All we will have then are the things we still believe in—our norms. But, by then, they might not be enough.

A central function of the United States Congress is oversight of the executive branch. Congressional oversight, as exercised from the beginning of the nation, is an essential tool in making the separation of powers real by empowering Congress to check the executive. In recent years, however, as polarization has reached paralyzing levels, Congress has largely gotten out of the business of routine and prospective “police-patrol” oversight.  In the absence of the will and the capacity to do prospective oversight, Congress is at risk of losing its power to the executive branch and thus failing one of its most important constitutional roles.

This paper assesses whether or not anything can be done to get Congress back into the oversight business. Specifically, author Elaine Kamarck examines the following question: Assuming that future Congresses develop the political will to conduct oversight, do they have the capacity to do oversight of a large, modern, and complex executive branch?

As Kamarck illustrates, mismatched resources may make it difficult for Congress to resume its oversight function. The modern federal government is a complex and enormous enterprise. But as the executive branch has grown considerably over the past decades, Congress has adopted budget cuts that make the legislative branch less and less capable of undertaking the kinds of systemic oversight that can solve or prevent problems. Congress employs a mere 17,272 professional staff to oversee an executive branch consisting of 4.2 million civil servants and uniformed military. 

“The existing infrastructure that is supposed to help Congress be on top of the executive branch has fallen prey to a mindless dumbing down of Congress,” Kamarck states. She details the five entities that are meant to support Congress in its oversight role: committee staff, the Congressional Research Service, the Government Accountability Office, the Congressional Budget Office, and the Inspectors General, all of which are understaffed and under-budgeted. Kamarck recommends the first thing Congress should do to fix its oversight problem is to properly staff the agencies it already has and to stop nickel and diming and degrading its own capacity.

Furthermore, Kamarck calls for a “Congressional Oversight Office,” a body charged with evaluating governmental performance before a crisis arises. This office should be staffed by implementation professionals who can gather the signals from all the other oversight organizations annually and in sync with the budget cycle.

“Congress needs to get back into the business of productive executive branch oversight,” concludes Kamarck. A Congressional Oversight Office is certainly a step in that direction.

Downloads

• Download the paper

There will always be winners and losers as the global economy shifts and evolves. For a long period in the mid- to late 20th century, those losers were cities. Across the developed world, suburbanization shrank inner-city populations just as the industrial base that had once fueled growth succumbed to globalization.

At the end of the 20th century, as global cities such as New York and London pulled themselves out of the malaise of the 1970s, economic growth still eluded many smaller, formerly industrial cities across the United States and Europe. Catalyzing recovery in those older industrial areas was the focus of a decade-long effort of the London School of Economics and the Brookings Institution. As is clear in Cities for a Small Continent, a new book from Anne Power at LSE, the potential in these cities is greater now than ever. In our contribution to the volume, we examine the why and the how of economic transformation in several U.S. cities. 

There has been a lot of focus on the shift in location preferences that is bringing people back to cities. Significant shares of millennials as well as empty nesters are voting for urban communities where they can live, work, and play. At least as important is the restructuring of the U.S. economy—from a closed innovation system where corporations operated isolated research facilities, to an open, networked economy where corporations innovate in collaboration with universities, researchers, entrepreneurs, and investors. Innovation is critical, because as Antoine van Agtmael and Fred Bakker assert in The Smartest Places on Earth, “the era of cheap [in manufacturing] is over; the era of smart has begun.”

These shifts in social preferences and market forces revalue cities and “cityness”—proximity, density, vibrancy, authenticity, and diversity. In particular, population and employment growth is occurring in downtowns and midtowns that have key institutions and assets: universities, medical campuses, cultural venues, historic buildings, walkable streets, and transit connectivity.

This regeneration is being delivered through a new localism in U.S. governance. Every day brings new bottom-up, city-led approaches to the training of workers, the education of children, the mitigation of climate change, the financing of infrastructure, and the development of affordable housing for our workers and quality places for our young and elderly populations.

Across this wide range of activity are some common characteristics.

Cities are harnessing the power of networks of government, business, civic, philanthropic, university, and community institutions and leaders rather than relying on public-sector solutions alone. The focus of the new American localism on unlocking the latent capacity and creativity of public, private, and civic networks differs markedly from the focus of traditional federalism on relationships between levels of government, particularly the federal government and the states.

Cities and metropolitan areas are also deploying capital from an array of public, private, and civic sources at the local, national, and even global levels. With federal investment dwindling, financing of critical projects will increasingly come from public-private collaboration and require experimentation around new forms of innovative finance.

Our chapter highlights four cities in the United States—Pittsburgh, Philadelphia, Cleveland, and Detroit—where this new localism has delivered tangible results. Though each city is at a different point of recovery, all have experienced growth in their cores that has been enabled and co-led by anchor institutions, major philanthropies, private-sector leaders, and civic groups. The biggest investments and decisions in these places have been the results of collaborative processes—proof that cities and the institutions that invest in them can be a source of long-term, strategic thinking that ultimately leads to healthier and more prosperous urban economies.

Similar efforts are spreading across the United Kingdom and Europe, though the systems there tend more toward public-sector leadership. In Sheffield, England, a concerted effort by business and academic institutions to “upskill” the manufacturing base, enabled by the flexibility of a “city deal” from the central government, has made the city a global center of advanced manufacturing. Bilbao, Spain evolved from a manufacturing base to a vibrant urban cultural hub by leveraging the value of publicly owned land and other assets for regeneration purposes. Stories such as these are featured throughout Cities for a Small Continent, as well as in a new series of seven case studies from LSE.

We are still in the early stages of this rebalancing of growth. Cities and metropolitan areas experienced decades of population and employment decentralization, poverty concentration, racial separation, and de-industrialization. Such patterns do not get changed overnight. But they are changing. As cities innovate, those solutions must be captured and codified and then replicated across the world.

Watch the May 24, 2016 LSE launch event for Cities for a Small Continent here: 

U.S. and EU officials have called on Turkey to show restraint amid concerns that Friday's failed coup attempt will be the pretext for an authoritarian crackdown. Turkey’s ability to investigate the putsch will be compromised by the fact that President Recep Tayyip Erdoğan has repressed the media and weakened the judiciary, says Kemal Kirisci, of the Brookings Institution. The coup attempt will likely bolster Erdoğan’s efforts to consolidate power, and that will make any investigation into the true causes of the coup difficult, Kirisci says.

President Recep Tayyip Erdoğan has pinned responsibility on the cleric Fethullah Gülen, who lives in self-imposed exile in the United States. Is that credible?

It is possible [that the Gülen movement is responsible], but this needs to be thoroughly investigated. The [Turkish] government has long argued that Fethullah Gülen leads a parallel state from Pennsylvania, where he has been living since 1997. The journalist and academic Ali Bayramoğlu has said that the movement, going back to the 1980s and early 1990s, began to systematically place their students in the police and military academies. These students were closely monitored and accompanied.

Other commentators have argued that it's unbelievable how F-16 pilots, on whom the state has spent more than a million dollars each to educate, attacked the Turkish parliament, military headquarters, intelligence services, and communications centers. How can a well-trained, prestigious F-16 pilot do this unless they are deeply attached to a cause? The movement also [includes] high-ranking judges and officers, including many one-star generals, who have been arrested.

Erdoğan has renewed his calls on the United States to extradite Gülen. Is this more plausible after the attempted coup?

Turkey has for a long time been seeking Gülen’s extradition, but the United States has not been cooperative, saying there was not enough evidence to build a case. However, the [U.S.-Turkey] extradition treaty refers to an obligation to extradite anyone who attempts to assassinate the leaders of one or the other country. There clearly was an attempt to assassinate Erdoğan: the hotel where he had been staying was attacked, and there was an attempt to control the airport where he was supposed to land. A necessary condition of extradition is that a convincing connection between the coup and the sought-after person is established; that should be the focus of cooperation.

Many analysts expect that Erdoğan will use the attempt as a pretense to invest the presidency with full executive powers, as he’s long sought.

The difficult issue will be to assess Erdoğan’s accusations independent of his political ends. Many commentators say the allegations put forward by the government serve his political aspirations: Erdoğan has become, de facto, the executive president of Turkey, but he wants to introduce an element of legality to it, which will require either a constitutional amendment or a completely new constitution. Opinion polls just before the coup suggested that the public, even supporters of [the ruling Justice and Development Party (AKP)], did not look favorably on it. This kind of threat to the system gives him considerable leverage [to codify an executive presidency], and I suspect he will follow that path rather than what some commentators are calling for, bringing the polarized country to a consensus through dialogue.

How to sort that out from the pursuit of the truth will be a difficult exercise, and I’m not sure today’s Turkey is equipped to do it, given the way the media has been repressed and the judiciary has been brought under the influence, if not control, of the government. These measures raise serious doubts about Turkey’s ability to investigate this within the realm of the rule of law.

The Incirlik air base, from which a U.S.-led coalition carries out air strikes against the self-proclaimed Islamic State, just reopened after a temporary closure. Will the coup attempt have broader implications for the anti-ISIS campaign?

Cooperation on ISIS is important, but much more so is the future of democracy in Turkey. A Turkey that is democratic is a Turkey of the rule of law, and a secular Turkey is one in which sympathy for such extremist groups is much less. Such a Turkey shares common values with the West, which makes cooperation much easier.

It looks like a large number of officers—colonels as well as one-star generals—were involved, and so the Turkish military is going to experience a period of weakness. That’s where U.S. cooperation, on intelligence and counterterrorism, is critical. Such cooperation demands trust. That trust will partially, but importantly, be a function of [cooperation over] the Gülenist dossier.

The United States and Turkey are going to have to find a way to cooperate over the PYD (Kurdish Democratic Union Party). [Editor’s note: the United States backs the Syrian Kurdish party’s militant wing, the YPG, to combat the Islamic State; Turkey says the group has aided its compatriots in Turkey, the PKK, whose insurgency against the state resumed in July 2015]. The United States enjoys leverage over the PYD, and now it’s ever more important that it keeps the PYD in line so it does not become involved in the conflict in Turkey and strengthen the hand of the PKK while Turkey is vulnerable.

Turkey hosts roughly three million refugees and the country is party to an agreement with the EU to stem migration to Europe. What will EU-Turkey relations look like in the aftermath of the coup attempt?

If the perpetrators had gained control of government, it would have led to civil war, so the very fact that the coup attempt collapsed is saving Europe from waves of Turkish asylum seekers. The EU will need to expand its basis of cooperation with Turkey, because this coup attempt will adversely impact Turkey’s ability to deal with these refugees. There will likely be a purge from some of the bureaucracies and organizations that deal with them.

More broadly, both U.S.-Turkey and EU-Turkey ties have been strained lately. Will the Turkey’s alliance with the West come out on stronger footing after their condemnation of the coup attempt?

Helping with the investigation to establish the truth will be critical. The way in which, at the end of the day, [the EU and United States came out] with support for the elected government, just as Turkish opposition parties did, should entitle them to some influence, calling on Turkey to live up to the norms, standards, and values of democratic society.

The EU and United States together must reanchor Turkey solidly within the Western alliance. Erdoğan, as much as [the United States and EU member-states] may resent him or disagree with policies, must be made to feel that he's welcome back in the ranks, the way he was between 2003 and 2010, on the condition he reforms his policies.

Erdoğan and the people around him, especially [former Prime Minister Ahmet] Davutoglu, had delusions of grandeur. They dreamed they were going to lead the ummah, the Islamic world. This is all gone now: the emperor is naked. Erdoğan seems to recognize Turkey has boxed itself into a corner and wants to come out in the direction of the West. There is nowhere else to go. Turkey needs tourism, international trade, and foreign investment, and to protect its national security. On all these grounds it needs cooperation, and it’s not going to come from Iran, China, or Russia, even if that might be their preference. It can only come from the West.

Turkey may also recognize that the West is in trouble too, and if the West cannot come out of that trouble, Turkey will be in even deeper trouble. Brexit is going to have a negative impact on Turkey's export market and tourism. An EU in trouble is not going to be able to give visa liberalization to Turkish nationals, which symbolically will anchor Turkey in the West, and is also important economically. If the EU and United States move to the right and say no to free trade agreements [and migration], Turkey will not benefit. Both sides next each other even more than in the past.

Copyright © Council on Foreign Relations 2016, republished with permission

Event Information

The failed coup in Turkey on July 15 to 16, organized by factions within the Turkish military in an attempt to overthrow the government of President Erdoğan, represents both a victory and a new trial for Turkish democracy. Although the Turkish citizenry brought the country back from the brink of anarchy and civil war, many analysts see last week’s events as a consequence of the political instability and discord that has been mounting for years as Erdoğan has consolidated powers, marginalized the opposition, and redefined Turkey’s democracy. How will the president react in the aftermath of the coup? Will the democratic backsliding intensify, or can the thwarted coup offer new opportunity for reconciling the deeply-polarized nation?

The upheaval and political instability in Turkey also holds significant implications for Turkey’s foreign policy and the fate of a neighboring region already in turmoil from the war in Syria and insecurity in Iraq. The West desperately needs a stable, democratic, and predictable partner in its NATO-ally Turkey to address the many challenges besetting the region and to fight the Islamic State (or ISIS). How will recent events affect regional stability and Turkey’s cooperation with the West on security issues, including the resettlement of Syrian refugees? What does the failed coup mean for the coalition against ISIS engagement in Syria?

On July 20, the Foreign Policy program (FP) at Brookings hosted a panel discussion to consider these questions and other domestic and international consequences of the coup attempt in Turkey. Brookings Senior Fellow and Director of the Center on the United States and Europe Fiona Hill introduced and moderated a wide-ranging conversation featuring FP Senior Fellows Shadi Hamid, Kemal Kirişci, Michael O'Hanlon, and Ömer Taşpınar.

After the discussion, the speakers took questions from the audience.

Video

• Turkey coup attempt was a shock
• Institutional collapse of Turkey unprecedented
• Disaster averted in failed Turkey coup
• Incirlik Air Base not irreplaceable
• Turkey after the coup attempt: Implications for Turkish democracy, foreign policy, and the future of the Syrian War

Audio

• Turkey after the coup attempt: Implications for Turkish democracy, foreign policy, and the future of the Syrian War

Transcript

• Transcript (.pdf)

Event Materials

• 20160720_turkey_coup_transcript

Editor's Note: The current U.S. presidential race demonstrates the deep political divisions that exist in our country. But what does it mean to be "liberal" or "conservative," "Republican" or "Democratic"? According to Shadi Hamid, certain values transcend political chasms. This post originally appeared on PBS NewsHour.

What does it mean to say that the Republican Party is on the “right”? The GOP, long defined (at least in theory) by its faith in an unbridled free market, the politics of personal responsibility, and a sort of Christian traditionalism, is no longer easily plotted on the traditional left-right spectrum of American politics. Under the stewardship of presidential nominee Donald Trump, the Republican Party appears to be morphing into a European-style ethnonationalist party. With Trump’s open disrespect for minority rights and the Bill of Rights, the GOP can no longer be considered classically “liberal” (not to be confused with capital-L American Liberalism). This is a new kind of party, an explicitly illiberal party.

These developments, of course, further constrain Republicans’ appeal to minority voters (I haven’t yet met an American Muslim willing to admit they’re voting for Trump, but they apparently exist). This makes it all the more important to distinguish between conservative values and those of this latest iteration of the Republican Party.

There are some aspects of Burkean conservative thought – including aspects of what might be called civic communitarianism – that could plausibly strike a chord in the current cultural landscape across “left” and “right,” categories which, in any case, are no longer as clearly distinguishable as they once were. (Take, for example, British Labour leader Jeremy Corbyn’s Euroskepticism and that of his opponents on the right, or the populist anti-elitism and trade protectionism that are now the province of both Republicans and Democrats).

Everyone seems angry or distrustful of government institutions, which, even when they provide much needed redistributive fiscal stimulus and services, are still blamed for being incompetent, inefficient, or otherwise encouraging a kind of undignified dependency. After the Brexit debacle, it seemed odd that some of the most Europhobic parts of Britain were the very ones that benefited most from EU subsidies. But this assumes that people are fundamentally motivated by material considerations and that they vote – or should vote – according to their economic interests.

If there’s one thing that the rise of Trump and Brexit – and the apparent scrambling of left-right divides – demonstrates, it’s that other things may matter more, and that it’s not a matter of people being too stupid to realize what’s good for them. As Will Davies put it in one of the more astute post-Brexit essays, what many Brexiteers craved was “the dignity of being self-sufficient, not necessarily in a neoliberal sense, but certainly in a communal, familial and fraternal sense.”

The communitarian instinct – the recognition that meaning ultimately comes from local communities rather than happiness-maximizing individuals or bloated nanny-states – transcends the Republican-Democratic or the Labour-Conservative chasm. In other words, an avowedly redistributive state is fine, at least from the standpoint of the left, but that shouldn’t mean neglecting the importance of local control and autonomy, and finding ways, perhaps through federal incentives, to encourage things like “local investment trusts.”

Setting up local investment trusts, expanding the child tax credit, or introducing a progressive consumption tax aren’t exactly a call-to-arms, and various traditionalist and communitarian-minded philosophers have, as might be expected from philosophers, tended to stay at the level of abstraction (authors armed with more policy proposals are more likely to be young conservative reformers like Ross Douthat, Reihan Salam, and Yuval Levin). Douthat and Salam want to use wide-ranging tax reform to alter incentives in the hope of strengthening families and communities. This is a worthy goal, but realizing such policies requires leadership on the federal level from the very legislators who we should presumably become less dependent on.

This is the reformer’s dilemma, regardless of whether you’re on the left or right. If your objective is to weaken a centralized, overbearing state and encourage mediating or “middle” institutions, then you first need recourse to that same overbearing state, otherwise the proposed changes are unlikely to have any significant impact on the aggregate, national level.

The fact that few people seem interested in talking about any of this in our national debate (we instead seem endlessly intrigued by Melania Trump’s copy-and-paste speechwriting) suggests that we’re likely to be stuck for some time to come. Incidentally, however, the Hillary Clinton campaign slogan of “Stronger Together” has an interesting communitarian tinge to it. I doubt that was the intent, and it’s only in writing this column that I even took a minute to think about what the slogan might actually mean. I, as it happens, have been much more interested in talking about – and worrying about – an unusually fascinating and frightening man named Donald Trump.

Five years ago, my mother needed an orthopedic surgeon for a knee replacement. Unable to find any data, we went with an academic doctor that was recommended to us (she suffered surgical complications). Last month, we were again looking for an orthopedic surgeon- this time hoping that a steroid injection in her spine might allay the need for invasive back surgery. This time, thanks to a recent data dump from CMS, I was able to analyze some information about Medicare providers in her area and determine the most experienced doctor for the job.  Of 453 orthopedic surgeons in Maryland, only a handful had been paid by Medicare for the procedure more than 10 times.  The leading surgeon had done 263- as many as the next 10 combined. We figured he might be the best person to go to, and we were right- the procedure went like clockwork.

Had it been a month prior to the CMS data release, I wouldn’t have had the data at my fingertips. And I certainly wouldn’t have found the most experienced hand in less than 10 minutes.

It’s been a couple of month since the release of Medicare data by the Centers for Medicare and Medicaid (CMS) on the volume and cost of services billed by healthcare providers, and despite the whiff of scandal surrounding the highest paid providers (including the now-famous Florida ophthalmologist that received $21 million) the analyses so far have been somewhat unsurprising. This week, coinciding with the fifth Health DataPalooza, is a good time to take stock of the utility of this data, its limitations, and what the future may hold.

The millions of lines of data was exactly as advertised: charges and paid services under traditional Medicare “fee-for-service,” including the billing provider’s ID and the costs to Medicare. The initial headlines touting “Medicare Millionaires” relied on some basic arithmetic and some sorting.  And the cautions piled up: the data could reflect multiple providers billing under a single ID; payments are not the same as a provider’s actual take home income; it’s not complete information as it doesn’t contain information about other insurers, or even Medicare Advantage, and so on.

But perhaps most damning was how little insight the data seemed to provide on the quality or value of care provided, as opposed to volume of services.  As Lisa Rosenbaum wrote in the New Yorker, “So much of that good isn’t captured by these numbers. You don’t bill for talking to a patient about how he wants to die. There’s no code for providing reassurance rather than ordering a test.”

Where is the value in the data?

Data bear witness to the fundamental flaw of the payment system that generates them. The absence of information on quality, safety, appropriateness, or outcomes appears to have been a genuine revelation to many, but it is in fact exactly the type of output that we should expect from this volume-based system that we have built. This is not a critique of the data release. It is an indictment of our payment system.

Data is revealing important trends in how we pay doctors differently. Not all physician payments are created equal, and the data certainly shows the disparities across specialties, primary care, and others. For example, the average total annual Medicare payment to geriatricians was less than $100,000, while dermatologists and radiation oncologists (who presumably also see non-elderly patients) received on average $200,000 and $360,000 respectively. The important question will be why and should it continue?

Figure 1: Distribution of Total Medicare Pay by Provider Type, 2012 

Source: Author's calculations based on Medicare data released in April 2014

Data is revealing important indicators of cost and pricing – a major contributor to rising health care costs. Why is it that a brief visit with a geriatrician is worth $13; a 45-minute visit with a geriatrician sorting through medications, educating family members, and developing a quality of life plan with a terminal cancer patient is worth $79; and a dermatologist treating suspected skin cancer can earn upwards of $600 for a procedure that takes them minutes?

Data sheds light on practice patterns. The data is also revealing important variances in utilization of drugs and treatments. For example, a block apart on Park Avenue, two ophalmologists differ significantly in their use of treatments for macular degeneration. One uses expensive injectable drugs and gets paid over $10,000 per injection, while the other receives less than $500 for the lower-cost equivalent.

A CBS News report looked at spinal fusion surgeries—a procedure where there is almost no evidence demonstrating a net benefit to patients compared to other conservative therapies. They observed that “while the average spine surgeon performed them on 7 percent of patients they saw, some did so on 35 percent.”

At the extremes, outlier “practice pattern” begins to raise questions of potential improper billing or outright fraud and abuse. For example, simply looking at the frequency and volume of services provided to individual beneficiaries can identify concerning outliers. This laboratory company billed for 28,954 blood glucose reagent strips in 2012- for 88 patients. And yes, that’s highly unusual.

Figure 2: "Outlier" Medicare Billing for Blood Glucose Reagent Strips, 2012

Source: Author's calculations based on Medicare data released in April 2014

One clinical social worker billed for 1,697 separate days of service on 28 patients (the size of the bubble is proportional to the total amount of reimbursement by Medicare in 2012).

Figure 3: "Outlier" Medicare Billing for Days of Service, 2012

Source: Author's calculations based on Medicare data released in April 2014

The most extreme outlier, Dr. Gary Ordog, was named by NPR and ProPublica in their examination of providers who are outliers on their pattern of coding for the highest intensity office. Ordog had previously lost the right to bill California’s state Medicaid program, and yet continued to charge Medicare for over $500,000 in billing in 2012. It’s important to caution however, that even in these extreme outliers, statistics alone cannot provide definitive evidence of abuse. There is a need for formal investigation.

Medicare and law enforcement officials will need to create new processes for dealing with a potential flood of outlier reports from amateur sleuths like me.

What's Next for Medicare Data?

Data can be trended. Updates of data releases can begin to show us not just snapshots, but moving pictures of our healthcare system as it undergoes rapid changes. The New York Times reported on the increase in charges for certain frequent causes of hospitalization between 2011 and 2012. It will be interesting to see whether the data release itself, and the Steven Brill landmark Time article on hospital charges, have an impact on reversing these trends.  

Data can be “mashed up”.  The value of open data is hugely greater than the sum of its parts. As more and more data becomes available, the files can be cross-linked and “mashed up” to be able to answer questions no one database could have.  ProPublica linked together cobbled together data on state actions and sanctions on physicians with the Medicare data release to ask why these physicians are still being paid by Medicare.

What does the future hold? Correlations with drug prescribing data, meaningful use, and referral patterns are possible today, Sunshine Act disclosures and quality reporting, and much more is soon to come.

As we get comfortable with the data, analysts can move past the basics of arithmetic and sorting, we have an opportunity to make more ‘meaningful use’ of this data. We can begin to identify practice patterns, overuse, variations in geography or demographics, and potentially even fraud and abuse. As more and more data becomes available, the files can be cross-linked and “mashed up” to be able to answer questions no one database could have addressed. What will determine the value of the Medicare data release will be the creativity of those data scientists, epidemiologists, and health services researchers (amateur as well as professional) who can ask the challenging questions that must be answered.

Contributors: Alice M. Rivlin and Christine Dang-Vu

Recent data suggest that Accountable Care Organizations (ACOs) are improving important aspects of care and some are achieving early cost savings, but there is a long way to go. Not all ACOs will be successful at meeting the quality and cost aims of accountable care. The private sector has to date allowed more flexibility in terms of varying risk arrangements—there are now over 250 accountable care arrangements with private payers in all parts of the country—with notable success in some cases, particularly in ACOs that have been able to move farther away from fee-for-service payments. Future growth of the Medicare ACO program will depend on providers having the incentives to become an ACO and the flexibility to assume different levels of risk, ranging from exclusively upside arrangements to partial or fully capitated payment models.

Given that the first three year cycle of Medicare ACOs ends in 2015 and more providers will be entering accountable care in the coming years, the Centers for Medicare and Medicaid Services (CMS) has indicated that they intend to release a Notice of Proposed Rulemaking (NPRM) affecting the Medicare ACO program.

In anticipation of these coming changes, the Engelberg Center for Health Care Reform has identified the "Top Eight ACO Challenges" that warrant further discussion and considerations for ensuring the continued success of ACOs across the country. To support that discussion, we also present some potential alternatives to current Medicare policies that address these concerns. These findings build on the experiences of the Engelberg Center’s ACO Learning Network members and other stakeholders implementing accountable care across the country.  In some cases, the alternatives might have short-term costs, but could also improve the predictability and feasibility of Medicare ACOs, potentially leading to bigger impacts on improving care and reducing costs over time.  In other cases, the alternatives could lead to more savings even in the short term. In every case, thoughtful discussion and debate about these issues will help lead to a more effective Medicare ACO program.

Top Eight ACO Challenges

1. Make technical adjustments to benchmarks and payments
2. Transition to more person-based payments
3. Increase beneficiary engagement
4. Enhance and improve alignment of performance measures
5. Enable better and more consistent supporting data
6. Link to additional value-based payment reforms
7. Develop bonus payments and other incentives to participate
8. Support clinical transformation

Downloads

• Issue Brief: Improving the Medicare ACO Program

A recent blog suggested that CMS’ efforts to standardize assessment data was based on a goal of “….creating a functional measurement tool that could be used throughout the industry.” In fact, CMS has been working since 2005 to meet the Congressional directive to standardize assessment information at hospital discharge, and post-acute care (PAC) admission and discharge for payment and quality reporting purposes (Deficit Reduction Act of 2005). The CARE tool was developed as part of the national Post-Acute Care Payment Reform Demonstration (PAC PRD). The conceptual domains and items were selected with the input of the wide range of stakeholder communities working with PAC populations. Clinicians from acute hospitals and each of the four PAC settings, including long term care hospitals (LTCHs), inpatient rehabilitation facilities (IRFs), skilled nursing facilities (SNFs), and home health agencies (HHAs) identified items to test in four areas: medical status, functional status, cognitive status, and some social support factors. Input was given by physicians, nurses, physical therapists, occupational therapists, speech and language pathologists, social workers and case managers working in each of the different levels of care. Initial item selection was based on a review of existing assessment items, including those in the three Federally-mandated instruments, (the IRF-PAI, MDS 2.0, and OASIS-B which were in effect at this time) and the input of each of the scientific communities working in these areas.

Developers of proprietary systems such as the UDS-MR©, Inter-RAI ©, and AM-PAC ©, as well as public domain items tested in clinical trials such as the PROMIS items, were all reviewed as part of this process. The selected items needed to be in the public domain so the measures could be modified as science advanced practice.

Over 200 providers participated nationwide to submit over 53,000 CARE assessments over the course of the PAC PRD. Participating clinicians also provided feedback during training and exit interviews. In general, positive feedback was provided on most items. Feedback showed that almost all items were commonly collected on existing instruments in hospitals and PAC providers, although some of the information may have been informally noted in charts rather than provided in the structured form of the CARE items.

The items were tested for reliability so they could be applied consistently across populations and settings. Most of the items were previously tested and found reliable in at least one of the five levels of care. Two types of reliability tests were conducted on the final CARE tool item set used in the PAC PRD. The results showed that most items when applied to the other four settings were at least as reliable as the existing Federal assessment items (Kappa scores of 0.6 or better) ensuring their reliable use in future quality measures or payment models would reach consistent results. Complete reports on item reliability and PAC PRD results can be found here.

Data standardization is critical to allow providers to exchange information as they follow the patient. The Deficit Reduction Act of 2005 directed CMS to use standardized assessment items at acute hospital discharge and PAC admission and discharge to allow for empirical comparisons of key questions arising out of changing incentives in the Medicare payment policies. The standardized CARE items are consensus-based versions of the items already collected by clinicians. These and additional items being incorporated into CMS’ assessment item library represent the “best in class.” The team developing the CARE item set represented the leading experts in each of the areas – Dr. Margaret Stineman of the University of Pennsylvania, developer of the function-related groups associated with the proprietary FIM©, Dr. Deborah Saliba, UCLA, lead developer of the MDS 3.0, and Dr. Chris Murtaugh of the Visiting Nurse Service of New York. Team members included Drs. Anne Deutsch and Trudy Mallinson of the Rehabilitation Institute of Chicago. Input was also given by Dr. David Hittle, of the University of Colorado who has worked closely with the OASIS tool, Dr. Samuel Markello, formerly of the UDS-MR©, and Dr. Patrick Murray of Case Western University.

The blog suggested that, “the early reviews of the CARE tool have been poor.” While this clearly is not true, it is worth pointing out that the author owns one of the key proprietary assessment instruments. The CARE items have been evaluated for reliability and they meet the national standards; they allow providers and others the opportunity to download the e-specification of the items without charge and to have the clinicians trained for free under CMS’ regular assessment training initiatives. CMS is currently developing quality measures using the “best in class” assessment items which all meet scientific standards. The quality measure development process already requires CMS to submit measures for endorsement by the National Quality Forum. The “loophole” identified by the UDS-MR© author is non-existent. The Measures Application Partnership is part of the existing NQF process included in the IMPACT legislation. Further, use of uniform data elements across settings, such as those used in the currently collected pressure ulcer measure, allows for exchangeability and improves communication across the system, finally creating a “data follows the person” system.

Erica has a history of cardiac issues. She visits her doctor for a regular checkup and her doctor writes a new prescription to better control her heart disease. Unfortunately, her doctor didn't mention any instructions, except to take it once a day. Erica thanks her doctor and heads to the pharmacy. At the check-out counter, the clerk hands Erica her new prescription drug, in addition to three documents stapled to the bag that he says "will explain everything you need to know about your medication." Later on, while reviewing the materials at home, Erica is overwhelmed by the information, which is in fine print and difficult to understand. She is frustrated and confused, and tosses the documents in the trash.

This scenario is not uncommon. Research suggests that about 50 percent of Americans find it difficult to read health information.[i] Consumers who cannot find the information they need, or who do not understand the information because it is presented in a convoluted manner, are less likely to use it to prevent unnecessary medical errors. In Erica’s case, she could have ended up in the emergency room because she missed some basic warnings about her prescription. For example, one warning might have been that she should not chew the medication because it was an extended release capsule. Chewing the capsule could release the entire day’s dose at once, resulting in an unintended overdose.

We know that consumers are receiving information – sometimes too much information. Not only are consumers receiving pages of medication information, the information they receive is uncoordinated and sometimes conflicting. Some documents are written by the drug manufacturer, and others are written by pharmacies or another third party. Some medication information documents are FDA-approved and others are not.

The real question is – could medication information be presented in such a way that it would be more useful for consumers? The answer is a resounding “yes.” One study found that just 75 percent of consumer medication information met the minimum criteria for usefulness.[ii] That number might be impressive as a field goal percentage in the NBA, but for consumers it represents an unmet need for high quality medication information.

The U.S. Food and Drug Administration (FDA) has spent the past several years working with stakeholders to determine the most effective methods for conveying medication information. One overarching principle that has emerged from FDA’s engagement with the health care community is the need for a single, standardized document to replace the numerous existing documents. This document is identified as Patient Medication Information (PMI).

PMI creates an easier way for consumers to access and understand their medication information. By presenting the most salient pieces of information – including drug uses, warnings, side effects, and directions – on a single page that is easy to navigate, PMI can be a useful tool for enhancing treatments and preventing avoidable medication errors or side effects. PMI holds promise both for consumers and the broader health care system. For consumers, PMI could contribute to better outcomes and an overall improvement in patient experience. For health systems, PMI’s positive impact on medication adherence could improve performance on quality measures, such as hospital readmissions, that could lead to shared savings or other rewards.

Through a cooperative agreement, the Engelberg Center for Health Care Reform at the Brookings Institution has worked in collaboration with FDA over the past few years to convene a series of workshops focused on identifying best PMI practices – for example, how to make PMI both more usable and accessible.  Workshop participants identified several guiding principles for improving the content, format, and distribution of PMI.

PMI Guiding Principles

PMI content should be consumer-friendly. Expert stakeholders identified a lack of consumer-friendly information as one of the most important barriers to effectively communicating critical medication information. To fix this problem, the language used in PMI will need to be simplified, patient-centric, and understandable across the entire spectrum of health literacy levels. The types of information that should be included in PMI must be essential for taking a medication properly. Extraneous information, such as a discussion of previous treatments a consumer must have previously tried and failed before receiving the new prescription, may be more confusing than helpful.

The best PMI formats are simple and easy to navigate. Consumers don’t want to be given a technical-looking instruction manual when they pick up their prescriptions. Participants at the workshops generally agreed that it would be ideal to keep PMI to a single page. They also agreed that actionable headers that help consumers locate the information they are looking for are preferable to the question and answer format (e.g., “Uses” and “Directions” are more effective than “What does the drug treat?” and “How do I use the drug?”). There was consensus on the point that consumers will ultimately decide the best format.

Access to PMI will be bolstered by multiple channels of distribution. Paper is still the primary source of medication information, and is preferred by certain demographics. However, technology is revolutionizing the way consumers receive information. This is generally good for society, but it introduces some challenges, including the fact that consumers now have more access to information of questionable quality.  One method for ensuring access to consistent and high quality PMI would be to have a central repository for all PMI documents. This approach could support distribution of both printed and electronic PMI. Access to PMI could be further enhanced by making it available on smartphones and via email.

On July 1, the Center will convene a public meeting that will provide an opportunity for the health care community to discuss the issues mentioned above. Researchers will give an update on progress made since the previous meetings and share the lessons they learned from recent studies. Diverse stakeholders – including patient advocacy groups, providers, pharmacies, and drug manufacturers – will provide their perspectives on the future of PMI and assess their role in making high quality PMI a reality. 

There are many issues that need to be addressed in exploring the promise of PMI. However, one thing that participants at the July 1 meeting should remember is this: Keep it simple, stakeholders.

Event Information

Many clinicians have terrific ideas for improving the quality and cost of health care, but often don’t know how to navigate the often baffling landscape of payment and delivery reform options. To address this need in clear, practical terms, we are pleased to announce the second MEDTalk event in the “Merkin Series on Innovations in Care Delivery.” The series is designed to support clinicians and policymakers who’ve always wondered how delivery reform occurs, but didn’t know where to begin. 

Our second case focused on the work of leaders from the New Mexico Cancer Center (NMCC), and their efforts to "Reinvent Patient-Centered Cancer Care." The event featured several brief “TED-style” talks that considered the challenges of delivering oncology care, while enhancing patient experience, improving coordination of care, and reducing costs. The agenda included firsthand experiences from patients, payers, policymakers, and NMCC's clinical leadership who explores sustainable improvement strategies, and the financial mechanisms available to encourage innovations in oncology.

Video

• A Day in the Life: The Patient Experience
• The Future of Oncology: Drugs, Genetic Testing and Personalized Medicine
• The Challenge of Delivering Cancer Care in a Triple Aim World
• What We're Learning from New Oncology Payment Models
• Supporting Care While Reducing Costs: What is Possible?
• The New Mexico Experience: Aligning Clinical Redesign and Payment Reform
• Sustaining Improvements in Patient-Centered Care
• Lessons Learned and the Path Forward
• MEDTalk: Reinventing Patient-Centered Cancer Care

Event Materials

• Oncology Case Study August 2014 FINAL WEB

ACOs have to assess both the opportunities and the challenges in pursuing risk based payment arrangements. However, there are key strategies ACOs can adopt to help with the transition to greater financial accountability, and higher levels of performance. On July 16^th, our ACO Learning Network hosted a webinar that explored critical success factors and barriers, as well as potential lessons for ACOs to increase their risk-bearing capacity.

Different Payment Models Pose Unique Challenges

Drawing upon an issue brief authored with colleagues from the American Academy of Actuaries, Greger Vigen highlighted some of the actuarial concerns inherent in various risk-based payment models. For example, when organizations undertake one-sided or “bonus only” shared savings arrangements, the arrangement should create incentives that are adequately aligned across the system to reward physicians that are providing not just cost-efficient, but also high-value, care. They must also find ways to create additional non-financial support when the size of provider shared savings is minimal. Both one-sided and two-sided shared savings models can lead to complex calculations in determining the true amount of savings resulting from the arrangement. Organizations undertaking bundled or episode-based payments must clearly determine what is included in specific bundles in order to justify assuming risk for those services.  Although partial capitation lends itself to an increased level of risk by creating strong incentives to reduce inefficiencies in certain parts of the system, such arrangements create effects on areas not covered by the partial capitation arrangement. Finally, by virtue of the increased risk level, global payment arrangements raise solvency considerations for organizations and require significant investment in infrastructure (or use resources from aligned partners) to manage utilization and facilitate appropriate organizational culture change. As programs move forward, there may be a combination of approaches used, including broad risk sharing or partial capitation arrangements between buyer and provider organizations. At the same time, more focused initiative may be used between the provider organization and the provider partners, such as bundled payments or partial capitation to certain providers. 

Critical Elements of High Performing Systems

Regardless of which risk-type arrangements ACOs choose to pursue, there are some critical elements that most high-performing systems share. According to Vigen, high-performing systems typically use a highly analytic process to understand their performance over time, rely on financial committees that are clear and “blunt” about the organization’s financial state, have in place multiple targeted financial initiatives, use payment reform to show how expenses could reduce revenue across multiple specific cases, have advanced clinical reporting systems, develop strong partnerships within and outside the organization to transform care, and use “next generation” analytic tools from outside organizations. Vigen contends that, in order to succeed at true transformation, organizations must develop initiatives aimed specifically at financial results, to supplement other initiatives designed to simply improve quality.  Building around these initiatives provides targeted results that can effectively engage and utilize actuaries. Actuaries and other external partners can help ACOs to more fully address financial issues and develop a framework for the prioritization and allocation of resources that identifies which existing processes to discontinue and which new processes to initiate.

Jim Whisler, a Principal at Deloitte Consulting, added that there are a number of opportunities to realize savings, many of which can be achieved through reducing variation. He emphasized that ACOs should take advantage of “low hanging fruit” that decrease utilization and optimize costs, such as ensuring appropriate use of generic and specialty pharmaceuticals, reducing inpatient stays, increasing use of ambulatory surgery center (ASC), and understanding appropriate use of lab, radiation, MRI, and CT scans. Savings and gain sharing can also vary between the different providers—primary care, specialists, and hospitals.  Ultimately, actuaries and clinical staff alike must be able to analyze data and pinpoint the true drivers of variation.

Effective Strategies Can Yield Significant Cost and Quality Improvements

In order to shed more light on how these strategies can be deployed in health systems, Bart Wald, Chief Executive for Physicians Services at Providence Health and Services, discussed the approaches that he undertook as President and CEO of Physician Associates of the Greater San Gabriel Valley, an IPA in California. Unlike compensation in most IPAs, which often relies either on fee-for-service (FFS) or capitation, Physician Associates, developed a system that uses a combination of both payment models. While capitation has traditionally been used more for primary care physicians (PCPs) than specialists, Dr. Wald emphasized that developing financial incentives for specialists was critical to effectively engaging them.  In his mind, specialists must be integrated and engaged in order to create a truly effective model. The IPA also created a “peer satisfaction” bonus program, in which PCPs and specialists rate each other and those attaining the highest ratings are given an additional bonus payment. The ratings are also posted on the web to increase transparency, and impact physician behaviors. In addition to engaging specialists, Physician Associates made sure to integrate hospitalists into the care team and incorporate social services and mental health staff.

The IPA was able to further transform care by building a population health infrastructure that directed funds towards ambulatory care management, patient-centered medical home (PCMH) development, complex care centers and disease management programs for vulnerable patients. They have also focused on improving the collection and sharing of patient data to better manage generic drug utilization, prescribing and adherence; track and attempt to minimize care received by patients outside of the physician network; and ensure effective continuity of care across the system. In summary, Dr. Wald stressed a number of factors for succeeding at risk-sharing—engaged practice members, adequate physician incentives to improve care performed in the outpatient setting, hospitalists and outpatient coordination on referrals, more advanced pharmacy management, extended disease registries, and integration between physicians and hospitals that includes joint expense management and other synergies. These interventions have resulted in a noticeable improvement in quality of care and led Physician Associates to run at a price and expense advantage below IPAs using a purely FFS payment model.

Dr. Wald also discussed an example of how hospitals are attempting to improve performance through creative partnerships with medical staff using a joint venture limited liability corporation between Providence Health Systems California and local physicians to reduce the occurrences of specific clinical conditions, such as hospital acquired complications, readmissions, and sepsis mortality. These physicians participate in a shared savings model.

Successful Risk-Sharing Arrangements Take Time

While the interventions described above may work and possibly work very well, truly effective risk-sharing design is an iterative process. Organizations must be nimble and able to adapt and modify tactics as necessary. As Greger Vigen emphasized, payment reform can be used as a tactic at the provider level in additional to a broad strategy or vision between provider and buyer.

Downloads

• Issue Brief: Improving the Medicare ACO Program

Please note that this Engelberg Center for Health Care Reform Health Policy Issue Brief first appeared in the Health Affairs Blog on July 31, 2014. Click here for the Health Affairs Blog version.

Abstract

Individual patient expanded access is a process by which patients can obtain investigational drugs that have not been approved by the Food and Drug Administration (FDA) outside of a clinical trial setting from biopharmaceutical companies when no other alternative therapy is available. Currently, no industry-wide structural principles exist to help companies navigate this process while balancing the needs of getting a drug to the market as quickly as possible with providing potentially life-saving treatment to individual patients. The Engelberg Center convened a stakeholder group to identify common themes and identify common principles related to expanded access, as none currently exist. The result was 4 A’s - Anticipation, Accessibility, Accountability, and Analysis – to help assist patients, providers, and companies with expanded access. Process and capacity building recommendations for the FDA also were proposed to assist companies with sustaining expanded access programs.

Call to Action: The Importance of Expanded Access Programs

Individual patient expanded access, sometimes termed “compassionate use,” refers to situations where access to a drug still in the development process is granted to patients on a case-by-case basis outside of a clinical trial, prior to completion of mandated clinical trials and approval by the Food and Drug Administration (FDA). This typically involves filing a single patient or emergency investigational new drug (IND) request with the Food and Drug Administration and voluntary release of the drug by the manufacturer. Generally, the following criteria must be met: there is reasonable expectation of meaningful benefit despite the absence of definitive clinical trial data, the patient has a serious or life-threatening condition, there are no comparable or satisfactory treatment alternatives, and there are no suitable clinical trials for the drug available to the patient. This form of expanded access, which is the focus of this paper, is different from the situation in which a drug is discharged to a large group of needy patients in the interval between successful phase 3 trials and presumed FDA approval, a strategy often termed a “treatment” IND or protocol, which was initially used in the 1980s for releasing zidovudine to patients with acquired immune deficiency syndrome.

The Engelberg Center for Health Care Reform at the Brookings Institution recently invited senior leaders from several pharmaceutical companies, two bioethicists, a senior FDA representative, and a patient advocate to share experiences and discuss organizational strategies related to expanded access (see acknowledgements). A driving factor for this meeting was a recent flurry of highly public cases of desperate patients seeking access to experimental drugs, which lead to social media campaigns and media coverage. Such cases included 7-year-old Josh Hardy (brincidofovir from Chimerix for disseminated adenovirus infection), 45-year-old Andrea Sloan (BMN673 from BioMarin for ovarian cancer), 41-year-old Nick Auden (pembrolizumab from Merck for melanoma), and 6-year-old Jack Fowler (intrathecal idursulfase from Shire for Hunter Syndrome). Expanded access requests to the FDA for new patients are increasing, from 1,000 patients nationwide in 2010 to more than 1,200 in 2012.[i] (This is likely an underestimate, since it does not include appeals made directly to companies.)

In the wake of these events, it became clear that many biopharmaceutical companies had varying experiences and policies related to such access. From the domestic regulatory standpoint, the FDA revised its expanded access regulations in 2009, which define criteria that must be met to authorize expanded access, list requirements for expanded access submissions, describe safeguards that will protect patients, and preserve the ability to develop meaningful data about the use of the drug. Biopharmaceutical companies typically face a complex global environment in which legal and regulatory frameworks can differ substantially. At the meeting, a senior FDA representative indicated the agency has approved over 99 percent of expanded access requests submitted via single patient or emergency INDs since 2009, suggesting the regulatory agency is not a major barrier to expanded access. As such, provided the access request is reasonably related to the potential benefits of the drug, the biopharmaceutical company is almost solely responsible for the decision and liability regarding whether to grant expanded access to an individual. Still, the public belief persists that the FDA is the main bottleneck that restricts access. In April 2014, Representative Morgan Griffith (R-VA) proposed H.R. 4475, The Compassionate Freedom of Choice Act of 2014, designed to restrict the FDA’s ability to prevent the use of investigational drugs in terminally ill patients. Similarly, some states have passed “Right to Try” legislation to reduce FDA oversight, but contains no requirement that companies must make drugs available.[ii]

The goal of our meeting was to identify common themes and possibly broad outlines to suggest industry-wide policies related to expanded access, as none currently exist. The group first discussed background issues related to expanded access and agreed on definitions. The meeting then focused on three topics. First, the group participants who play key roles in evaluating expanded access requests were invited to share narrative experiences in specific clinical cases, in an effort to lay the groundwork for trust and open discussion. Second, the group was asked to identify internal industry-specific structural barriers, such as the existence of clear procedures or tracking mechanisms within companies to handle requests. Finally, the participants reflected on situations in which expanded access may not be appropriate, or where regulatory barriers or liability concerns may hinder expanded access. This paper reflects the authors’ observations and assessment of the internal and external landscape, based upon information provided by the meeting participants.

Laying the Groundwork with Shared Experiences

The FDA allows companies to provide drugs and charge individual patients that do not meet the enrollment criteria for clinical trials geared towards regulatory approval through expanded access programs.[iii] These programs are meant to provide the drug directly to treat the patient’s condition, rather than having the primary goal of collecting efficacy or detailed safety data in support of approval. Before 1987, the FDA lacked formal recognition of expanded access, although investigational drugs were provided informally.[iv] Since then, the FDA has instituted novel classes of individual INDs so that a company sponsor or licensed physician can legally obtain treatment access from the FDA to provide a drug while it is still in the approval process.[v] Essentially, this provides companies a legal exception from the law to ship unapproved drugs across state lines, and if they desire, to charge for them. These INDs are designed solely for the potential benefit of desperate patients and not intended to formally collect safety or efficacy data that could potentially inform a regulatory decision, but can have regulatory impact, nonetheless.

At the outset, several participants objected to the term “compassionate use,” since it introduces inherent value decisions, can emotionally charge discussions, and does not recognize that there may be valid and ethically appropriate reasons for denial. The generally agreed upon term “expanded access,” is used throughout this paper. (One participant suggested the term “early access.”) Ideally, the term would make it obvious that this is access to an unapproved drug, in order to temper expectations of favorable results. Somewhat confusingly, the FDA uses the terms “expanded access,” “access,” and “treatment use” interchangeably to refer to the use of a drug, and of which none clearly identify the stage of development.[vi]

Participants shared numerous examples of requests for expanded access and explained that their companies handle anywhere from a handful to several hundred requests per year. The following selected stories illustrate the wide range of experiences and situations that companies encounter when navigating the complex decisions involved in administering an expanded access program. Several other examples were discussed and the specific participants expressed that they would be willing to share these particular examples publicly.

Chimerix, a 54-employee company based in Durham, North Carolina, is developing the drug brincidofovir and previously had created an intermediate expanded access protocol for the drug (CMX001-350) as encouraged by the FDA following over 200 emergency INDs granted for access to brincidofovir.[vii] One such case was for an armed services member with previously undiagnosed acute myelogenous leukemia who developed life-threatening vaccinia infection following smallpox vaccination in 2009.[viii] The patient received the drug from Chimerix through an emergency IND. After two years, the company had not secured FDA approval for the drug and eliminated expanded access in February 2012 in order to focus on studies which would inform a regulatory decision. In March 2014, Chimerix originally rejected an emergency IND request for 7-year old, Josh Hardy, who was critically ill from disseminated adenovirus infection after bone marrow transplantation. A highly public social media campaign targeted the company in the wake of this decision, and the experience was traumatizing for many of the employees. Following discussion with the FDA, Chimerix initiated a new clinical trial for the treatment of adenovirus infection in order to collect safety and efficacy data to support an NDA submission. Hardy was the first patient enrolled in the clinical trial, and his family reported through several media outlets that he recovered from the adenovirus infection and was discharged home.

One biopharmaceutical company representative described receiving a middle-of-the-night telephone call directly at home, with an emergent, time-sensitive request for an experimental therapy for a critically ill child with a rare acute disease in a foreign pediatric intensive care unit, where regulatory standards were different from those in the U.S. The ideal pediatric dosage was unknown, and only limited safety data and clinical details were available. Urgent efforts were made to gather more information and the request was approved, but despite these efforts the patient did not survive.

Bristol-Myers Squibb began a clinical trial for a cancer drug several years ago.[ix] A woman with pancreatic cancer enrolled in the trial and saw that her tumor was no longer growing. After the 3.5 year trial, the study closed because the drug was deemed ineffective for all other patients and was not approved for further development. However, the company continued to provide the drug for the one woman for whom the drug was effective through a single patient IND for an additional 9 years.

To demonstrate the volume of expanded access requests, one participant showed several messages on his mobile device during the half-day discussion, directly from patients who had located his email addresses through on-line searches, to plead for expanded access to an anticancer therapy.

Development of Structural Principles: The Four A's 

Broadly, no specific industry-wide consensus on expanded access procedures exists. As a result, there is significant variation in company policies and procedures. During this phase of discussion, participants shared their own company strategies and suggested possible areas of consensus that might form the basis for shared principles and industry-wide practices. These suggestions fell into four categories, which we termed the 4 “A’s”: Anticipation, Accessibility, Accountability, and Analysis (see Figure 1).

First, the group agreed that large and small companies should anticipate the need for and creation of expanded access programs when developing drugs expected to generate expanded access requests and as part of the drug development plan. This is particularly important for drugs that might be considered for priority or breakthrough designation during FDA approval. In these cases, companies should strongly consider developing a written expanded use policy with clear guidelines for inclusion and exclusion, which would also feature a defined review process, clear decision making criteria, and a defined time frame for response to requests. This also allows companies to plan for the demands that may be placed on their supply chain and staff resources to ensure sufficient supply for investigational and expanded use purposes. Identifying a decision maker within each company and for each disease area/product will also help patients or physicians reach the appropriate contact when requesting a drug, as well as assist the company in gaining expertise in responding to these requests. For example, one large company identifies one point of contact for all expanded access requests regarding each product and posts that individual’s contact information on the website.

In the early stages of drug development, supplies of investigational drugs are extremely limited. This is often because the technically-challenging process of optimizing drug product manufacture takes a considerable amount of time. Low yielding manufacture batches are not uncommon at the early phases of research. Some companies do not approve expanded access requests because they do not have enough of the drug in stock to supply these external requests and meet the needs of investigational study patients and individuals participating in clinical trials, an issue which may be particularly acute for biologics. Smaller companies may have more resource constraints, such as inadequate staff to manage requests or supply chain and logistics issues. One representative suggested that if a company had early transparency from regulators about the final numbers of subjects they would be willing to accept to achieve drug development milestones, it would make it much easier for the company to feel less reservation about its drug supply. (It may be beneficial for companies to analyze their financial ability to provide drugs potentially at no cost or when there is not a large enough supply, ideally in a transparent manner.)

Once an expanded access policy is anticipated and developed, the second key principle the group identified was making the policy accessible to all individuals who may qualify. First, for patients, with guidance from their treating physician, the company making the drug should always steer the patient to enter a clinical trial (if they meet eligibility criteria). If the contacted company cannot accommodate the patient, they should steer them to other open trials if possible, even if sponsored by another company. Many of our participants noted that this already occurs.

The group was particularly cognizant of the disparity in access to drug companies and their expanded access programs: patients with savvy social media strategies are more likely to succeed in navigating across organizational constraints than without similar sophistication. The group believes that increased accessibility would assist in making opportunities for expanded access more equitable. In addition, these policies could help educate patients and physicians about submitting legitimate expanded access requests and help decrease the costs of reviewing inappropriate requests on the company (for example, if there are other proven therapies or the situation is not life threatening).

If the patient is ineligible for a trial, the patient should be able to easily access the written expanded access policy online. For example, both large and small companies like Pfizer, Bristol-Myers Squibb, Shire, and Merck post their expanded access policies on their websites, though the terminology may in some cases be complex. In addition, Janssen has developed a video explaining their policies in non-technical terms. Ideally, such policies should be available in some web based or public facing platform to both patients and physicians and written in a clear manner that is jargon free and accessible to individuals at various education levels. Most participants felt strongly that requests for expanded access should originate from a medical provider, not from a patient, since expertise is needed to first screen appropriate candidates. This is consistent with current FDA regulations for an IND, in which a physician or qualified medical expert must sponsor an IND or serve as an investigator under an existing IND for expanded access.

Third, companies should have accountability to the requesting party for expanded use requests that they receive and review them within a specified, transparent amount of time. If the request could not be approved, the company should consider clear communication and provide an explanation of why the request was turned down. In these cases, some participants suggested that the company might also consider instituting an appeals process by which a patient can receive an additional review if not approved, potentially from a non-binding third party such as an independent, multidisciplinary body or a regulatory agency like the FDA. (Two participants, however, were uncomfortable with any third party review.)

The final principle would encourage companies to release timely analysis of data from expanded access patients. In addition to tracking communication, companies should keep a database of the number of requests and outcomes, in a manner that doesn’t slow getting drugs to needy patients rapidly. One company refined its internal tracking tools to determine who was requesting drugs, for what conditions, and where they lived. Where possible, companies might be encouraged to share anecdotal or preliminary safety or efficacy data from expanded access in peer-reviewed or other refereed venues in a prudent time frame following collections, if this is available or known. This is not always possible, because emergency INDs do not require provision of safety or outcome data to the company.

There are several challenges associated with operationalizing this in the current model, namely the appropriateness of anecdotal data, the level of detailed safety and efficacy data currently available through expanded access, suitability for publication, and funding for these activities in the current budget climate. One potential approach to address this is funding from federal or state regulatory agencies or payers for the reasonable costs of follow-up and reporting outcomes.

Regulatory Considerations

The participants then discussed the types of risks, including regulatory and financial, that may affect companies’ expanded access policies. When a company is considering expanded access requests, they consider the risks-benefits of providing the drug outside of a clinical trial as well as the potential for any regulatory issues in an era of litigation and an increased threshold for demonstration of safety. While a company’s provision of a drug for expanded access is voluntary, the FDA does require the company to collect and report safety data. Notably, none of the representatives felt that the FDA is a major regulatory barrier to processing and approving expanded access requests once the sponsor has reviewed the request, assessed the benefit-risk, and determined the request meets FDA requirements and evidentiary standards. In addition, the attendees felt that adverse effects and related liability risk were not of particular concern given that the drugs are assessed on a risk-benefit analysis.

However, companies that make drugs in particularly limited markets with small numbers of patients (for example, for unusual diseases with less than 200,000 patients nationwide which may justify a special designation called “orphan status”) may be more concerned about restrictive labeling if an unusual adverse event occurred even in one or two patients during expanded access of an orphan or small market therapy. However, there is no data of which participants were aware and no public reports that an adverse event during expanded access has harmed regulatory approval.[x] The group opinion was that that safety data would be available eventually in any event and an FDA “safe harbor” provision would not necessarily affect companies’ willingness to accept more requests for expanded access. A final concern was that there is no regulatory mechanism to consider data from expanded access in the evidence generation process for approval.

An Expanded Role for the FDA

While the FDA may not serve as a strong barrier to expanded access, the group considered strategies to promote equitable and fair access. For example, some argued that the breakthrough or priority review categories for FDA review might identify products that could have high potential for expanded access requests. This designation expedites “the development and review of drugs for serious or life-threatening conditions.”[xi] As of mid-April 2014, the FDA had received nearly 180 requests for breakthrough designation, with 44 requests granted.[xii] By hastening the drug development process, the FDA has already begun to bring drugs that have a reasonable expectation of benefit to the market faster. In order to receive breakthrough therapy designation, current legislation might be amended so companies could be asked to provide evidence that the 4 A’s are being followed in some capacity.

The FDA might also assist companies in establishing expanded access programs during open clinical trials in two main areas: process and capacity building. First, in terms of process, the FDA could be asked to create a defined path for regulatory approval with provisions that would encourage companies, both large and small, to include plans for expanded access programs when developing a drug. While FDA’s draft guidance related to INDs notes that larger expanded access programs could threaten enrollment in clinical trials,[xiii] and some participants agreed that this was a significant issue, not all companies have had difficulties enrolling patients in both clinical trials and expanded access programs. For example, one large pharmaceutical company left a Phase 1 clinical trial open for a promising therapy while concurrently enrolling individuals who didn’t qualify for open clinical trials into an expanded access program, without appreciable leakage of enrollees in their advanced phase trials that might affect the key development pathway.

Second, the FDA could support convening around capacity building and sharing best practices with companies. With the understanding that there are many small biotechnology or pharmaceutical companies with limited budgets and staff, the FDA could foster a partnership of large and small companies. This partnership could be achieved by convening meetings where companies share their experiences in creating and sustaining expanded access programs. This could be supported by creating a database for these shared ideas, as well as any expanded access data that can be made legally available, such as how many requests are granted or patient outcomes.

To ensure equitable, consistent, and transparent review of requests, some companies suggested the use of an impartial external advisory board. Similar to an unbiased review from an institutional review board (IRB), this committee could have an advisory or decision making function. Companies with supply constraints may feel that if they cannot give the drug to everyone who requests it, then they should give it to no one. This committee could help the company triage the patients who would benefit the most, and would be protected from liability.

Next Steps

The most efficient and equitable way to make new effective treatments to the largest number of needy patients is regulatory approval, accelerated or otherwise, following successful demonstration of efficacy and safety for a given indication in a specific population. Until that process is complete, access to an experimental therapy is by definition an additional risk, as the agreed necessary safety and efficacy have not yet been demonstrated. True informed consent in this setting is difficult to obtain (i.e. studies have shown that severely ill patients, such as those with life-threating circumstances requesting expanded access, had less retention of information discussed in the informed-consent process and less-clear understanding of the risks of therapy compared to healthier patients[xiv]).

One position companies and regulators can consider is that the default answer to expanded access requests should be affirmative, unless there are compelling reasons for not approving requests to patients with life-threatening illnesses. (Such reasons, for example, might include limited treatment supply or lack of reasonable expectation of benefits versus risks.) Such a position would require, however, that there be broader industry, clinician, regulatory, and patient advocacy agreement of shared principles. This paper outlines the experiences, structural principles, and regulatory considerations of a small group, but further meetings may convene a broader group of stakeholders to build upon these concepts. Such consensus-based approaches might lead to durable systems that meet the needs of desperate patients who have run out of options—while allowing innovation to continue to benefit those who may come afterwards.

Acknowledgements: We are grateful for the participation of the following representatives in the roundtable: Jeff Allen (Friends of Cancer Research), Michelle Berrey (Chimerix), Renzo Canetta (Bristol-Myers Squibb), Anne Cropp (Pfizer), Joseph Eid (Merck), Aaron Kesselheim (Harvard Medical School), Howard Mayer (Shire), Jeffrey Murray (FDA), Lilli Petruzzelli (Novartis), Amrit Ray (Janssen), and Robert Truog (Harvard Medical School). We thank Mark McClellan (Brookings Institution) for helpful discussions of this topic and comments on the manuscript, and to the Richard Merkin Foundation for support. The views and opinions expressed in this article were interpreted and organized by the staff of the Brookings Institution. They do not necessarily reflect the official policy or position of any individual roundtable representative, their companies, or their employers.

References

Downloads

• The Four A's of Expanded Patient Access

It is well recognized across the health care industry that the major goals of the Affordable Care Act (ACA) include not only expanding health insurance coverage, but also improving the quality of care and the patient health care experience. A key strategy in achieving these goals is improving the efficiency and delivery of care through innovative financing mechanisms and new delivery models, such as Accountable Care Organizations (ACOs), patient-centered medical homes (PCMHs), bundled payments for acute and post-acute care, and population-based models that aim to improve the health of entire communities. These alternative models emphasize quality and outcomes, while moving care away from the traditional and predominant method of fee-for-service (FFS).¹

The Frontline Work Force
Many conversations focused on the implementation of these models typically emphasize the role of physicians. However, the success of these models relies heavily on the support and manpower of a multidisciplinary team; particularly "frontline health care workers." Frontline workers may include medical assistants (MAs), medical office assistants, pharmacy aides, and health care support workers. Oftentimes, they provide routine, critical care that does not require post-baccalaureate training.²

For example, MAs can play an important role in a medical home model. Upon discharge from the hospital, frontline workers can provide direct outreach to patients that are at high risk for readmission, and discuss any lingering symptoms, worsening of conditions, or medication issues. If necessary, MAs can assign a high-risk patient to a social worker, care coordinator or nurse.³

In a team care environment, frontline health care workers are essential for taking over routine tasks and allowing physicians to employ their specialized skills on their most complex patient cases, which allows all team members to work at “the top of their license”.⁴ Frontline workers can also bridge the gap between patients and a multitude of providers and specialists; help deliver care that is culturally and linguistically appropriate; and provide critical patient education and outreach outside of regular office visits. 

A Workforce in Need of Reform
While team-based care is widely accepted as an industry norm, its current infrastructure is not well-supported. While the frontline workforce represents nearly half of all health care professionals, they are markedly underpaid, underappreciated, and lack formal training to transition into higher-skilled and/or higher paid positions.

A recent study by the Brookings Metropolitan Policy Program “Part of the Solution: Pre-Baccalaureate Healthcare Workers in a Time of Health System Change” demonstrates this glaring disparity between current frontline workforce investment and its value to health reform efforts. The study analyzes the characteristics of the top ten ‘pre-baccalaureate health care workers’ (staff that holds less than an associate’s degree) within the US’s one-hundred largest metropolitan areas (see Table 1).

Table 1: Top ten pre-baccalaureate health care workers in the US’s top one-hundred metropolitan areas

Personal care aides represent a striking example of the underinvestment in frontline workers. The study shows that personal care aides have the lowest levels of educational attainment compared to their peers (32% have no more than a high school diploma), and have the lowest median earnings ($20,000 annually). Meanwhile, The Center for Health Workforce Studies’ (CHWS) estimates that this profession is among the top three national occupations with the highest projected job growth between 2010 and 2020. They are also in highest demand: between 2010 and 2020 there will be an estimated 600,000 personal aide vacancies.⁵ According to this study, MAs are also among the least educated and lowest paid frontline professions. Ninety percent lack a bachelor’s degree and a significant share (29%) are classified as ‘working poor.’

Policy Solutions

A number of policy solutions can be applied to enhance the frontline worker infrastructure. Our recommendations include:

Invest in front line health care workforce training and education. Case studies from a recent Engelberg Center toolkit, outlines how providers are training their frontline workforce to master fundamental skills including care management, patient engagement, teamwork, and technological savviness.

For example, a New Jersey ACO carried out clinical transformation by investing in new frontline staff, and by redefining the role of medical assistants to include health coaching. The return on investment for employers is potentially large. After injecting a substantial initial investment into this project, this ACO saw a 12.3% decrease in net health care costs within the first year of the program’s implementation; as well as significantly improved efficiency, quality of care and patient experience. As the educational curricula for frontline professions are largely variable, more attention should also be spent on the quality of educational content to train these occupations, as well as on developing an understanding of how delivery systems are augmenting traditional educational curricula.

2. Active inclusion of frontline health care workers in payment reform. Although the services of frontline health care workers are beginning to play a role in new payment models, typically frontline staff does not benefit directly from any bonus payments or shared savings incentives. However, their increasingly valuable role in the care team may warrant allowing frontline health care staff to be included in the receipt of shared savings and/or bonus payments based on the achievement of specifically tailored performance and outcomes targets.

The increasing demand for frontline health care workers, driven in part by the ACA’s payment and delivery reforms, will likely spell out a brighter future for these occupations, whose services had routinely been undervalued and underpaid. Future policy efforts should be focused on extending educational grants that have been aimed at primary care and nursing to frontline workers, as well as considering dedicating portions of shared savings to enhancing the earning potential for frontline workers. Some efforts, such as the U.S. Department of Labor’s recent rule to grant wage and overtime protections to home health and personal care aides, are early suggestions of a shift toward greater respect and empowerment for these occupations. It is yet to be seen what effects the continuation of such efforts will have on their high projected attrition trends.

Event Information

The Brookings Institution and the American Enterprise Institute (AEI) collaborated to ask: if you were to redesign Medicare without spending more money, what would you keep and what would you change? A new report on a Center for Healthcare Decisions program provided insight into the public’s willingness to restructure Medicare in the face of tightening budget constraints. Using an interactive, computer-based system, program participants faced the challenge of making Medicare more responsive to the needs of current and future beneficiaries.

Were participants willing to accept limits on their choice of provider or reduced coverage of low-value medical care? Would they accept the need for greater personal responsibility in their use of health services? Would they agree that Medicare should adopt other policies to promote fiscal responsibility?

Watch event video.