Leaders will head to the G-20 Summit in Los Cabos, Mexico, among renewed serious concern about the world economy. The turmoil that started with the U.S. subprime mortgage crisis has resulted in now almost five years of ongoing instability. The emerging market economies fared much better than the advanced economies and pulled out of the crisis already in 2009, but the slowdown we are now facing in 2012 is again global, demonstrating the interdependence in the world economy. The emerging market economies have stronger underlying trend growth rates, but they remain vulnerable to a downturn in the advanced economies. The center of concern is now squarely on Europe, with a recession threatening most European countries, even those that had reasonably good performances so far. After an encouraging start in 2012, the U.S. economy, while not close to a recession, is also showing signs of a slowdown rather than the hoped for steady acceleration of growth. And the slowdown is spreading across the globe.

At a time like this it would be desirable and necessary that the G-20 show real initiative and cohesion. The essays in this collection look at the challenge from various angles. There is concern that the G-20 is losing its sense of purpose, that cohesion is decreasing rather than increasing, and that policy initiatives are reactive to events rather than proactive. Let us hope that at this moment of great difficulty, the G-20 will succeed in giving the world economy a new sense of direction and confidence. It is much needed.

Download » (PDF)

Editor’s Note: The following piece is a chapter from the 2012 edition of Eurasian Development Bank’s Eurasian Integration Yearbook.

INTRODUCTION

For centuries Central Asia was in the backwater of global political and economic attention, tales of “Great Games” and “Silk Roads” notwithstanding. However, interest in Central Asia from outside the region has been on the rise in recent years: Central Asia’s energy resources are of great importance to its neighbours in Europe and Asia. In addition, China wants a peaceful backyard, while Russia considers Central Asia part of its historical economic and regional interests and draws heavily on Central Asia migrants. Turkey is attracted by the common Turkic heritage of the region. Iran shares language and cultural ties with the Tajik people. The Central Asia’s Islamic tradition connects it with the Middle East and other Islamic countries. And now NATO countries rely on Central Asia for transit of their nonlethal military supplies in their engagement in Afghanistan.

There is wide agreement that economic prosperity and political stability in Central Asia is critical not only for the 60-plus million inhabitants of the region, but also for Central Asia’s neighbours, since Central Asia serves as a strategically important land bridge between Europe and Asia. Since the five Central Asian countries are landlocked small economies, a critical prerequisite for long-term economic growth and political stability is successful economic integration underpinned by effective regional cooperation.

This paper therefore addresses the central question of what are the prospects for regional economic integration and regional cooperation in Central Asia. It starts by briefly reviewing the role of Central Asia in the context of the overall process of Eurasian continental economic integration. It then considers what are the benefits and obstacles of regional integration and cooperation in Central Asia against the backdrop of lessons of international experience with regional integration and cooperation, and looks at four of the most important recent regional cooperation initiatives. In closing, the paper provides an answer to the question whether regional integration and cooperation in Central Asia are for real or only a mirage.

Downloads

• Download the full paper

Editor's Note: Johannes Linn discusses the potential of multilateral development banks in the latest G-20 Research Group briefing book on the St. Petersburg G-20 Summit. Read the full collection here.

The origins of the multilateral development banks (MDBs) lie with the creation of the World Bank at Bretton Woods in 1944. Its initial purpose, as the International Bank for Reconstruction and Development, was the reconstruction of wartorn countries after the Second World War. 

As Europe and Japan recovered in the 1950s, the World Bank turned to providing financial assistance to the developing world. Then came the foundation of the InterAmerican Development Bank (IADB) in 1959, of the African Development Bank (AfDB) in 1964 and of the Asian Development Bank (ADB) in 1966, each to assist the development of countries in their respective regions. The European Bank for Reconstruction and Development (EBRD) was set up in 1991, following the collapse of the Soviet Union, to assist with the transition of countries in the former Soviet sphere. 

The MDBs are thus rooted in two key aspects of the geopolitical reality of the postwar 20th century: the Cold War between capitalist ‘West’ and communist ‘East’, and the division of the world into the industrial ‘North’ and the developing ‘South’. The former aspect was mirrored in the MDBs for many years by the absence of countries from the Eastern Bloc. This was only remedied after the fall of the Bamboo and Iron curtains. The latter aspect remains deeply embedded even today in the mandate, financing pattern and governance structures of the MDBs. 

Changing global financial architecture 

From the 1950s to the 1990s, the international financial architecture consisted of only three pillars: the International Monetary Fund (IMF) and the MDBs represented the multilateral official pillar; the aid agencies of the industrial countries represented bilateral official pillar; and the commercial banks and investors from industrial countries made up the private pillar. 

Today, the picture is dramatically different. Private commercial flows vastly exceed official flows, except during global financial crises. New channels of development assistance have multiplied, as foundations and religious and non-governmental organisations rival the official assistance flows in size. 

The multilateral assistance architecture, previously dominated by the MDBs, is now a maze of multilateral development agencies, with a slew of sub-regional development banks, some exceeding the traditional MDBs in size. For example, the European Investment Bank lends more than the World Bank, and the Caja Andina de Fomento (CAF, the Latin American Development Bank) more than the IADB. There are also a number of large ‘vertical funds’ for specific purposes, such as the International Fund for Agricultural Development and the Global Fund to Fight AIDS, Tuberculosis and Malaria. There are  specialized trust funds, attached to MDBs, but often with their own governance structures.

End of the North-South divide 

Finally, the traditional North-South divide is breaking down, as emerging markets have started to close the development gap, as global poverty has dropped and as many developing countries have large domestic capacities. This means that the new power houses in the South need little financial and technical assistance and are now providing official financial and technical support to their less fortunate neighbors. China’s assistance to Africa outstrips that of the World Bank.

The future for MDBs 

In this changed environment is there a future for MDBs? Three options might be considered: 

1. Do away with the MDBs as a relic of the past. Some more radical market ideologues might argue that, if there ever was a justification for the MDBs, that time is now well past. In 2000, a US congressional commission recommended the less radical solution of shifting the World Bank’s loan business to the regional MDBs. Even if shutting down MDBs were the right option, it is highly unlikely to happen. No multilateral financial institution created after the Second World War has ever been closed. Indeed, recently the Nordic Development Fund was to be shut down, but its owners reversed their decision and it will carry on, albeit with a focus on climate change. 

2. Carry on with business as usual. Currently, MDBs are on a track that, if continued, would mean a weakened mandate, loss of clients, hollowed-out financial strength and diluted technical capacity. Given their tight focus on the fight against poverty, the MDBs will work themselves out of a job as global poverty, according to traditional metrics, is on a dramatic downward trend. 

Many middle-income country borrowers are drifting away from the MDBs, since they find other sources of finance and technical advice more attractive. These include the sub-regional development banks, which are more nimble in disbursing their loans and whose governance is not dominated by the industrial countries. These countries, now facing major long-term budget constraints, will be unable to continue supporting the growth of the MDBs’ capital base. But they are also unwilling to let the emerging market economies provide relatively more funding and acquire a greater voice in these institutions.

Finally, while the MDBs retain professional staff that represents a valuable global asset, their technical strength relative to other sources of advice – and by some measures, even their absolute strength – has been waning. 

If left unattended, this would mean that MDBs 10 years from now, while still limping along, are likely to have lost their ability to provide effective financial and technical services on a scale and with a quality that matter globally or regionally. 

3. Give the MDBs a new mandate, new governance and new financing. If one starts from the proposition that a globalised 21st-century world needs capable global institutions that can provide long-term finance to meet critical physical and social infrastructure needs regionally and globally, and that can serve as critical knowledge hubs in an increasingly interconnected world, then it would be folly to let the currently still considerable institutional and financial strengths of the MDBs wither away.

• The mandate of the MDBs should be adapted to move beyond preoccupation with poverty eradication to focus explicitly on global and regional public goods as a way to help sustain global economic growth and human welfare. Moreover, the MDBs should be able to provide assistance to all their members, not only developing country members. 

• The governance of the MDBs should be changed to give the South a voice commensurate with the greater global role it now plays in economic and political terms. MDB leaders should be selected on merit without consideration of nationality. 

• The financing structure should be matched to give more space to capital contributions from the South and to significantly expand the MDBs’ capital resources in the face of the current severe capital constraints.

In addition, MDB management should be guided by banks’ membership to streamline their operational practices in line with those widely used by sub-regional development banks, and they should be supported in preserving and, where possible, strengthening their professional capacity so that they can serve as international knowledge hubs. 

A new MDB agenda for the G20 

The G20 has taken on a vast development agenda. This is fine, but it risks getting bogged down in the minutiae of development policy design and implementation that go far beyond what global leaders can and should deal with. What is missing is a serious preoccupation of the G20 with that issue on which it is uniquely well equipped to lead: reform of the global financial institutional architecture. 

What better place than to start with than the MDBs? The G20 should review the trends, strengths and weaknesses of MDBs in recent decades and endeavour to create new mandates, governance and financing structures that make them serve as effective pillars of the global institutional system in the 21st century. If done correctly, this would also mean no more need for new institutions, such as the BRICS development bank currently being created by Brazil, Russia, India, China and South Africa. It would be far better to fix the existing institutions than to create new ones that mostly add to the already overwhelming fragmentation of the global institutional system.

Event Information

In 2015, 193 countries adopted the Sustainable Development Goals (SDGs), a new global agenda that is more ambitious than the preceding Millennium Development Goals and aims to make progress on some of the most pressing issues of our time. Goal 4, "To ensure inclusive and quality education for all, with relevant and effective learning outcomes," challenges the international education community to meet universal access plus learning by 2030. We know that access to primary schooling has scaled up rapidly over previous decades, but what can be learned from places where transformational changes in learning have occurred? What can governments, civil society, and the private sector do to more actively scale up quality learning?

On April 18-19, the Center for Universal Education (CUE) at Brookings launched "Millions Learning: Scaling Up Quality Education in Developing Countries," a comprehensive study that examines where learning has improved around the world and what factors have contributed to that process. This two-day event included two sessions. Monday, April 18 focused on the role of global actors in accelerating progress to meeting the SDGs. The second session on Tuesday, April 19 included a presentation of the Millions Learning report followed by panel discussions on the role of financing and technology in scaling education in developing countries.

 Join the conversation on Twitter #MillionsLearning

Video

• Getting millions to learn: What will it take to accelerate progress on meeting the Sustainable Development Goals?
• Scaling quality education: The launch of the Millions Learning report
• Do funders help or hinder scaling in education?
• What role can technology play in scaling education?

Audio

• Getting millions to learn: What will it take to accelerate progress on meeting the Sustainable Development Goals?

Transcript

• Uncorrected Transcript - Day 1 (.pdf)
• Uncorrected Transcript - Day 2 (.pdf)

Event Materials

• 20160418_millions_learning_transcript
• 20160419_millions_learning_transcript

I had been mulling a rebuttal to my colleague and friend Jon Rauch’s interesting—but wrong—new Brookings paper praising the role of “hacks, machines, big money, and back room deals” in democracy. I thought the indictments of Chris Christie’s associates last week provided a perfect example of the dangers of all of that, and so of why Jon was incorrect. But in yesterday’s L.A. Times, he beat me to it, himself defending the political morality (if not the efficacy) of their actions, and in the process delivering a knockout blow to his own position.

Bridgegate is a perfect example of why we need fewer "hacks, machines, big money, and back room deals" in our politics, not more. There is no justification whatsoever for government officials abusing their powers, stopping emergency vehicles and risking lives, making kids late for school and parents late for their jobs to retaliate against a mayor who withholds an election endorsement. We vote in our democracy to make government work, not break. We expect that officials will serve the public, not their personal interests. This conduct weakens our democracy, not strengthens it.

It is also incorrect that, as Jon suggests, reformers and transparency advocates are, in part, to blame for the gridlock that sometimes afflicts our American government at every level. As my co-authors and I demonstrated at some length in our recent Brookings paper, “Why Critics of Transparency Are Wrong,” and in our follow-up Op-Ed in the Washington Post, reform and transparency efforts are no more responsible for the current dysfunction in our democracy than they were for the gridlock in Fort Lee. Indeed, in both cases, “hacks, machines, big money, and back room deals” are a major cause of the dysfunction. The vicious cycle of special interests, campaign contributions and secrecy too often freeze our system into stasis, both on a grand scale, when special interests block needed legislation, and on a petty scale, as in Fort Lee. The power of megadonors has, for example, made dysfunction within the House Republican Caucus worse, not better.

Others will undoubtedly address Jon’s new paper at length. But one other point is worth noting now. As in foreign policy discussions, I don’t think Jon’s position merits the mantle of political “realism,” as if those who want democracy to be more democratic and less corrupt are fluffy-headed dreamers. It is the reformers who are the true realists. My co-authors and I in our paper stressed the importance of striking realistic, hard-headed balances, e.g. in discussing our non-absolutist approach to transparency; alas, Jon gives that the back of his hand, acknowledging our approach but discarding the substance to criticize our rhetoric as “radiat[ing] uncompromising moralism.” As Bridgegate shows, the reform movement’s “moralism" correctly recognizes the corrupting nature of power, and accordingly advocates reasonable checks and balances. That is what I call realism. So I will race Jon to the trademark office for who really deserves the title of realist!

I this week had the pleasure of doing a podcast debate with my Brookings colleague Jonathan Rauch on the question of whether we need stronger machines and weaker transparency in American government, or the opposite. Guess which side I took! This has been a long-running water cooler and cafeteria discussion between Jon and myself since I arrived at Brookings almost a year ago. While we find some areas of agreement in the podcast (more than you might think),I remain unconvinced by the so-called “political realist” school that Jonathan is a leader of. As I have previously written and blogged (here, here and here), I think the realists are fantasists, disconnected from the actual reality of politics, including its risks. We need more transparency, not less to deal with, for example, things like corruption risk, particularly in the post-Citizens United era. Indeed, that decision itself embraces the value of a vigorous transparency regime when other safeguards are relaxed. My belief is that Washington works both more efficiently and more ethically under the scrutinizing gaze of the American media, ngo's and public. As former White House ethics czar, I often facilitated administration openness efforts, including as a means of accountability, for example helping put the White House visitor logs online.

Jon and my lively debate covers not only issues of transparency itself but also applies them to other current topics—the Affordable Care Act, Trade Promotion Authority, and much more. The debate was silently moderated by our colleague Ben Wittes as part of his “Chess Clock Debates” series. With only ten minutes on the chess clock each to make our points, it was a concise discussion that hit the fundamentals briskly. Thanks to Ben for inviting us and giving us a public forum to discuss this critical policy issue.

Editor's Note: : We’re hosting a conversation on Markaz on the Iranian nuclear talks, debating the merits of a deal, as well as the broader issues at stake for the United States and the region. This piece originally appeared in The Huffington Post.

As we near what may be the endgame of the current negotiations with Iran, I am reminded of the place where President Obama announced the overarching strategy that helped produce this moment: Prague. After stating his readiness to speak to Iran in a Democratic primary debate in 2007, and following that up postelection in 2009 with a series of initial statements directed to the Iranians, the president chose the Czech capital to lay out his vision of dealing with the dangers of nuclear weapons in April 2009. That included emphasizing that Iran would not be permitted to obtain a nuclear weapon on his watch: "Rules must be binding. Violations must be punished. Words must mean something. The world must stand together to prevent the spread of these weapons."

As a result of that 2009 speech, the president's nuclear strategy became known as the Prague Agenda. I had the privilege to travel with President Obama back to Prague in April 2010 to witness the signing of a major accomplishment in another area under the Prague Agenda, namely the New START treaty. By the following year, April 2011, I was in Prague as U.S. ambassador. That year, and in the each year that followed, we held an annual Prague Agenda conference to assess the steps that had been taken and the challenges that lay ahead.

In the years since, there has been steady progress in each of the four main areas the president laid out on that spring day in Prague in 2009. New START was a step forward on his first objective, to reduce the risks posed by existing nuclear weapons. Another goal, preventing nuclear terror by safeguarding materials and improving safety, has since been the subject of a series of successful Nuclear Security Summits in Washington, Seoul, and The Hague.

I saw first-hand the president's personal commitment to a third objective articulated in Prague: to promote the peaceful use of nuclear energy. I was, for example, present in the Oval Office in October 2011 when the president and the Czech prime minister met. President Obama advocated for the use of civil nuclear power as a part of the Czech energy mix (and also to achieve energy independence from Russia). That approach has been replicated in administration policy supporting civil nuclear energy in the United States and around the world.

Now, with the possible Iran deal, progress under the Prague Agenda's final prong is in reach: holding to account a state which had violated its nuclear obligations under international treaties. I am not of the school that believes the president needs to secure an Iran deal to build his legacy. That was never the case; having known him for almost a quarter of a century, since we were law students together, and having worked for him for six years, first in the White House and then as ambassador, I can attest that those kinds of considerations do not enter into critical decisions like this one. Even the president's strongest critics have to admit that legacy is, as a matter of logic, much less of a consideration after the recent breakthroughs on the Affordable Care Act and on Trade Promotion Authority.

Instead, as the comprehensive nature of the Prague Agenda itself suggests, President Obama is pursuing a deal out of principle. He is acting from his conviction that a good agreement with Iran represents another step toward making the U.S., our allies, and the world safe from nuclear terror. It is that ambition that has driven the president's formulation and consistent pursuit of each of the four elements of the Prague Agenda, the obscure aspects just as much as the headline-making ones.

Of course, as the president himself has repeatedly emphasized, the deal must be a good one. That is why I recently joined a bipartisan group of experts convened by the Washington Institute for Near East Policy in signing a statement laying out criteria for what any deal with Iran must at a minimum contain in five core areas: monitoring and verification; possible military dimensions; advanced centrifuges; sanctions relief; and consequences of violations. We also agreed on the importance of complementing any agreement with a strong deterrence policy and a comprehensive regional strategy. I have been encouraged by the warm reception for our statement from all corners, and by the strong tone struck by the American negotiators in Vienna this week. They recognize that willingness to walk away is the surest path to securing a good deal. If such a deal can be struck that meets the criteria in our bipartisan statement, that will be another stride forward under the Prague Agenda — perhaps the biggest yet.

With just over six months to go until the Iowa caucuses, news organizations are already speculating about what the Bush or Clinton or Trump Sanders administrations might look like. Though it might seem premature, their impulse is the right one. After the winner of the 2016 presidential race is announced in November of that year, the new President-elect will have just under three months to build his or her new government. From choosing cabinet members and key White House staff to setting the policy agenda and dealing with unanticipated crises, the presidential transition process is a huge undertaking, and one that requires much more advance planning than it is usually given.

Acknowledging the short timetable that surrounds the presidential transition process, on July 31 the Senate passed the “Edward ‘Ted’ Kaufman and Michael Leavitt Presidential Transitions Improvements Act of 2015.” If passed by the House and signed into law, this bill would require the president to establish a “White House Transition Coordinating Council” six months prior to the presidential election. This council would work with transition representatives for both candidates to prepare for the challenges that will lie ahead.

Under the new bill, the President would also be tasked to create an “Agency Transition Director’s Council.” This council would ensure that federal agencies function effectively through the transition. Again, transition representatives for each candidate would work with a group of senior representatives from the agencies, planning leadership changes and identifying potential obstacles.

Additionally, agency directors would designate “acting officers” for all essential non-career positions. In the event that these positions become vacant during the transition, a career civil servant from the agency will take over as “acting officer” until a replacement is appointed.

The Bush to Obama transition was one of the smoothest in history, and this bill reflects the best practices learned from that experience. (Full disclosure: one of the co-authors of this blog, Eisen, was the deputy general counsel of the Obama transition.) The Bush administration was ready early to work with the transition teams for both major party candidates. It offered a model of organization and cooperation with both campaigns well before Election Day. Once the election was decided, that engagement intensified, with constant contact and seamless teamwork between President-elect and his team and President Bush and his. Indeed, even after Election Day, many Bush appointees were asked to and did stay on longer in order to give the administration more time to find suitable replacements (See, e.g. Burke, p.594).

The Obama administration will undoubtedly "pay it forward" and meet those same high standards in addressing the upcoming transition. Nevertheless, codifying recent best practices as law makes eminent sense now, while we are all paying attention to the upcoming election—and knowing a future administration may not be as cooperative unless required by law. Although Inauguration Day 2017 may seem far off, there is actually not a moment to spare for this important legislation to proceed.

I had the pleasure earlier this month of welcoming my friend, Czech Republic Foreign Minister Lubomir Zaoralek, here to Brookings for a discussion of critical issues confronting the Europe-U.S. alliance. Foreign Minister Zaoralek was appointed to his current position in January 2014 after serving as a leading figure in the Czech Parliament for many years. He was accompanied by a distinguished delegation that included Dr. Petr Drulak of the Foreign Ministry, and Czech Ambassador Petr Gandalovic. I was fortunate enough to be joined in the discussion by colleagues from Brookings including Fiona Hill, Shadi Hamid, Steve Pifer, and others, as well as representatives of other D.C. think tanks. Our discussion spanned the globe, from how to respond to the Syrian conflict, to addressing Russia’s conduct in Ukraine, to the thaw in U.S.-Cuba relations, to dealing with the refugee crisis in Europe. The conversation was so fascinating that the sixty minutes we had allotted flew by and we ended up talking for two hours—and we still just scratched the surface.

Amb. Eisen and FM Zaoralek, October 2, 2015

Yesterday, we had a visit from Czech State Secretary Tomas Prouza, accompanied by Ambassador Martin Povejsil, the Czech Permanent Envoy to the EU. We also talked about world affairs. In this case, that included perhaps the most important governance matter now confronting the U.S.: the exceptionally entertaining (if not enlightening) presidential primary season. I expressed my opinion that Vice President Biden would not enter the race, only to have him prove me right in his Rose Garden remarks a few hours later. If only all my predictions came true (and as quickly). We at Brookings benefited greatly from the insights of both of these October delegations, and we look forward to welcoming many more from every part of the Czech political spectrum in the months ahead.

Prouza, Eisen, Povejsil, October 21, 2015

I was honored today to speak at the kick off meeting of the new ReFormers Caucus. This group of over 100 former members of the U.S. Senate, the House, and governors of both parties, has come together to fight for meaningful campaign finance reform. In the bipartisan spirit of the caucus, I shared speaking duties with Professor Richard Painter, who was the Bush administration ethics czar and my predecessor before I had a similar role in the Obama White House. 

As I told the distinguished audience of ReFormers (get the pun?) gathered over lunch on Capitol Hill, I wish they had existed when in my Obama administration role I was working for the passage of the Disclose Act. That bill would have brought true transparency to the post-Citizens United campaign finance system, yet it failed by just one vote in Congress.  But it is not too late for Americans, working together, to secure enhanced transparency and other campaign finance changes that are desperately needed.  Momentum is building, with increasing levels of public outrage, as reflected in state and local referenda passing in Maine, Seattle and San Francisco just this week, and much more to come at the federal, state and local level.

The European Union (EU) is confronting a series of potentially existential threats, including the refugee crisis, ISIS terror, Russian adventurism, and Brexit (the potential exit of the U.K. from the EU).  I hosted Czech Prime Minister Bohuslav Sobotka at Brookings to get his fundamentally (but carefully) optimistic take on how he and his fellow EU leaders can meet those challenges. Here are five reasons for optimism that emerged from our conversation: 

1. Take the Fight to Daesh.  The PM made clear Europe’s determination to take on the terror and refugee issues at their source in Iraq, Syria, and Libya.  Just this week, the Czech Republic upped its commitment to the international coalition, announcing that it will send a team to train Iraqis using U.S. made L-159 fighter jets (also sold to Iraq by Prague).  With transatlantic leadership, these efforts are starting to bear fruit in the decay of ISIS.
2. Never Let a Good Crisis Go to Waste. As part of addressing today’s refugee crisis, Europe is exploring multi-lateral efforts to construct a common European border service, integrate refugee populations, and promote internal security.  The process is painful, but filling these gaps will make the European Union stronger.
3. Stand Strong With Ukraine.  Some predicted that European unity against Putin’s expansionism would not hold.  Instead, the EU and the United States have maintained their resolve in enacting sanctions.  That has strengthened the EU, but as the PM pointed out, now Ukraine and its supporters must make sure that state moves towards good governance and functionality. 
4. Taking the Exit Out of Brexit.  The PM predicted that the U.K. would not exit the EU.  When I pressed him on why, he acknowledged that there were elements of wishing and hoping in that forecast, and that the vote comes at a tough moment.  But I share the PM’s hopes—the U.K. is not one to leave friends when times get tough.
5. Never Forget to Remember.  The PM and I spent a lot of time discussing the ups and downs of Central Europe’s experiment with democracy over the past century.  He and his Czech colleagues—of all mainstream political parties—are acutely aware of that history, and that too gives me hope that it will not be repeated.

Immense challenges can destabilize and divide—but they also present opportunities for new collaboration and cohesion. If addressed in partnership, Europe’s current trials can ultimately strengthen the ties that bind the EU together.  

Watch the full discussion here.

Andrew Kenealy contributed to this post. 

Event Information

A conversation with Prime Minister of the Czech Republic Bohuslav Sobotka

Today, the European Union faces critical risks to its stability. The possibility of a Brexit. The ongoing Ukraine/Russia conflict. The strain of mass migration. ISIL and other terrorism threats. The lingering financial crisis in Greece and beyond. These issues pose distinct challenges for the EU, its 28 member countries, and their 500 million citizens. How will these developing problems affect Europe?          

On March 31, Governance Studies at Brookings hosted Czech Prime Minister Bohuslav Sobotka to discuss the current status of the EU as seen through the lens of a Central European nation, close U.S. NATO ally and current Chair of the Visegrad Group. Prime Minister Sobotka offered insight into how the EU will address these issues, and where its future lies.

Video

• Challenges to the future of the EU: A Central European perspective

Audio

• Challenges to the future of the EU: A Central European perspective (English)

Transcript

• Transcript (.pdf)

Event Materials

• 20160331_czech_eu_transcript

How has the Joint Comprehensive Plan of Action (JCPOA)—signed between the P5+1 and Iran one year ago—played out in practice? Several Brookings scholars, many of whom participated prominently in debates last year surrounding official congressional review, offered their views.

Editor's Note: The introduction to the 2015 Brown Center Report on American Education appears below. Use the Table of Contents to navigate through the report online, or download a PDF of the full report.

TABLE OF CONTENTS

Part I: Girls, Boys, and Reading

Part II: Measuring Effects of the Common Core

Part III: Student Engagement

INTRODUCTION

The 2015 Brown Center Report (BCR) represents the 14^th edition of the series since the first issue was published in 2000.  It includes three studies.  Like all previous BCRs, the studies explore independent topics but share two characteristics: they are empirical and based on the best evidence available.  The studies in this edition are on the gender gap in reading, the impact of the Common Core State Standards -- English Language Arts on reading achievement, and student engagement.

Part one examines the gender gap in reading.  Girls outscore boys on practically every reading test given to a large population.  And they have for a long time.  A 1942 Iowa study found girls performing better than boys on tests of reading comprehension, vocabulary, and basic language skills.  Girls have outscored boys on every reading test ever given by the National Assessment of Educational Progress (NAEP)—the first long term trend test was administered in 1971—at ages nine, 13, and 17.  The gap is not confined to the U.S.  Reading tests administered as part of the Progress in International Reading Literacy Study (PIRLS) and the Program for International Student Assessment (PISA) reveal that the gender gap is a worldwide phenomenon.  In more than sixty countries participating in the two assessments, girls are better readers than boys. 

Perhaps the most surprising finding is that Finland, celebrated for its extraordinary performance on PISA for over a decade, can take pride in its high standing on the PISA reading test solely because of the performance of that nation’s young women.  With its 62 point gap, Finland has the largest gender gap of any PISA participant, with girls scoring 556 and boys scoring 494 points (the OECD average is 496, with a standard deviation of 94).   If Finland were only a nation of young men, its PISA ranking would be mediocre.

Part two is about reading achievement, too. More specifically, it’s about reading and the English Language Arts standards of the Common Core (CCSS-ELA).  It’s also about an important decision that policy analysts must make when evaluating public policies—the determination of when a policy begins. How can CCSS be properly evaluated? 

Two different indexes of CCSS-ELA implementation are presented, one based on 2011 data and the other on data collected in 2013.  In both years, state education officials were surveyed about their Common Core implementation efforts.  Because forty-six states originally signed on to the CCSS-ELA—and with at least forty still on track for full implementation by 2016—little variability exists among the states in terms of standards policy.  Of course, the four states that never adopted CCSS-ELA can serve as a small control group.  But variation is also found in how the states are implementing CCSS.  Some states are pursuing an array of activities and aiming for full implementation earlier rather than later.  Others have a narrow, targeted implementation strategy and are proceeding more slowly. 

The analysis investigates whether CCSS-ELA implementation is related to 2009-2013 gains on the fourth grade NAEP reading test.  The analysis cannot verify causal relationships between the two variables, only correlations.  States that have aggressively implemented CCSS-ELA (referred to as “strong” implementers in the study) evidence a one to one and one-half point larger gain on the NAEP scale compared to non-adopters of the standards.  This association is similar in magnitude to an advantage found in a study of eighth grade math achievement in last year’s BCR.  Although positive, these effects are quite small.  When the 2015 NAEP results are released this winter, it will be important for the fate of the Common Core project to see if strong implementers of the CCSS-ELA can maintain their momentum.

Part three is on student engagement.  PISA tests fifteen-year-olds on three subjects—reading, math, and science—every three years.  It also collects a wealth of background information from students, including their attitudes toward school and learning.  When the 2012 PISA results were released, PISA analysts published an accompanying volume, Ready to Learn: Students’ Engagement, Drive, and Self-Beliefs, exploring topics related to student engagement.

Part three provides secondary analysis of several dimensions of engagement found in the PISA report.  Intrinsic motivation, the internal rewards that encourage students to learn, is an important component of student engagement.  National scores on PISA’s index of intrinsic motivation to learn mathematics are compared to national PISA math scores.  Surprisingly, the relationship is negative.  Countries with highly motivated kids tend to score lower on the math test; conversely, higher-scoring nations tend to have less-motivated kids. 

The same is true for responses to the statements, “I do mathematics because I enjoy it,” and “I look forward to my mathematics lessons.”  Countries with students who say that they enjoy math or look forward to their math lessons tend to score lower on the PISA math test compared to countries where students respond negatively to the statements.  These counterintuitive finding may be influenced by how terms such as “enjoy” and “looking forward” are interpreted in different cultures.  Within-country analyses address that problem.  The correlation coefficients for within-country, student-level associations of achievement and other components of engagement run in the anticipated direction—they are positive.  But they are also modest in size, with correlation coefficients of 0.20 or less. 

Policymakers are interested in questions requiring analysis of aggregated data—at the national level, that means between-country data.  When countries increase their students’ intrinsic motivation to learn math, is there a concomitant increase in PISA math scores?  Data from 2003 to 2012 are examined.  Seventeen countries managed to increase student motivation, but their PISA math scores fell an average of 3.7 scale score points.  Fourteen countries showed no change on the index of intrinsic motivation—and their PISA scores also evidenced little change.  Eight countries witnessed a decline in intrinsic motivation.  Inexplicably, their PISA math scores increased by an average of 10.3 scale score points.  Motivation down, achievement up.

Correlation is not causation.  Moreover, the absence of a positive correlation—or in this case, the presence of a negative correlation—is not refutation of a possible positive relationship.  The lesson here is not that policymakers should adopt the most effective way of stamping out student motivation.  The lesson is that the level of analysis matters when analyzing achievement data.  Policy reports must be read warily—especially those freely offering policy recommendations.  Beware of analyses that exclusively rely on within- or between-country test data without making any attempt to reconcile discrepancies at other levels of analysis.  Those analysts could be cherry-picking the data.  Also, consumers of education research should grant more credence to approaches modeling change over time (as in difference in difference models) than to cross-sectional analyses that only explore statistical relationships at a single point in time. 

Downloads

• Download the report

Part I of the 2015 Brown Center Report on American Education. 

Girls score higher than boys on tests of reading ability.  They have for a long time.  This section of the Brown Center Report assesses where the gender gap stands today and examines trends over the past several decades.  The analysis also extends beyond the U.S. and shows that boys’ reading achievement lags that of girls in every country in the world on international assessments.  The international dimension—recognizing that U.S. is not alone in this phenomenon—serves as a catalyst to discuss why the gender gap exists and whether it extends into adulthood.

Background

One of the earliest large-scale studies on gender differences in reading, conducted in Iowa in 1942, found that girls in both elementary and high schools were better than boys at reading comprehension.^[i] The most recent results from reading tests of the National Assessment of Educational Progress (NAEP) show girls outscoring boys at every grade level and age examined.  Gender differences in reading are not confined to the United States.  Among younger children—age nine to ten, or about fourth grade—girls consistently outscore boys on international assessments, from a pioneering study of reading comprehension conducted in fifteen countries in the 1970s, to the results of the Program in International Reading Literacy Study (PIRLS) conducted in forty-nine nations and nine benchmarking entities in 2011.  The same is true for students in high school.  On the 2012 reading literacy test of the Program for International Student Assessment (PISA), worldwide gender gaps are evident between fifteen-year-old males and females.

As the 21^st century dawned, the gender gap came under the scrutiny of reporters and pundits.  Author Christina Hoff Sommers added a political dimension to the gender gap, and some say swept the topic into the culture wars raging at the time, with her 2000 book The War Against Boys: How Misguided Feminism is Harming Our Young Men.^[ii] Sommers argued that boys’ academic inferiority, and in particular their struggles with reading, stemmed from the feminist movement’s impact on schools and society.  In the second edition, published in 2013, she changed the subtitle to How Misguided Policies Are Harming Our Young Men.  Some of the sting is removed from the  indictment of “misguided feminism.”  But not all of it.  Sommers singles out for criticism a 2008 report from the American Association of University Women.^[iii] That report sought to debunk the notion that boys fared poorly in school compared to girls.  It left out a serious discussion of boys’ inferior performance on reading tests, as well as their lower grade point averages, greater rate of school suspension and expulsion, and lower rate of acceptance into college.

Journalist Richard Whitmire picked up the argument about the gender gap in 2010 with Why Boys Fail: Saving Our Sons from an Educational System That’s Leaving Them Behind.^[iv] Whitmire sought to separate boys’ academic problems from the culture wars, noting that the gender gap in literacy is a worldwide phenomenon and appears even in countries where feminist movements are weak to nonexistent.  Whitmire offers several reasons for boys’ low reading scores, including poor reading instruction (particularly a lack of focus on phonics), and too few books appealing to boys’ interests.  He also dismisses several explanations that are in circulation, among them, video games, hip-hop culture, too much testing, and feminized classrooms.  As with Sommers’s book, Whitmire’s culprit can be found in the subtitle: the educational system.  Even if the educational system is not the original source of the problem, Whitmire argues, schools could be doing more to address it. 

In a 2006 monograph, education policy researcher Sara Mead took on the idea that American boys were being shortchanged by schools.  After reviewing achievement data from NAEP and other tests, Mead concluded that the real story of the gender gap wasn’t one of failure at all.  Boys and girls were both making solid academic progress, but in some cases, girls were making larger gains, misleading some commentators into concluding that boys were being left behind.  Mead concluded, “The current boy crisis hype and the debate around it are based more on hopes and fears than on evidence.”^[v]

The analysis below focuses on where the gender gap in reading stands today, not its causes.  Nevertheless, readers should keep in mind the three most prominent explanations for the gap.  They will be used to frame the concluding discussion.

Biological/Developmental:  Even before attending school, young boys evidence more problems in learning how to read than girls.  This explanation believes the sexes are hard-wired differently for literacy.

School Practices: Boys are inferior to girls on several school measures—behavioral, social, and academic—and those discrepancies extend all the way through college.  This explanation believes that even if schools do not create the gap, they certainly don’t do what they could to ameliorate it. 

Cultural Influences: Cultural influences steer boys toward non-literary activities (sports, music) and define literacy as a feminine characteristic.  This explanation believes cultural cues and strong role models could help close the gap by portraying reading as a masculine activity. 

Table 1-1 displays the most recent data from eight national tests of U.S. achievement.  The first group shows results from the National Assessment of Educational Progress Long Term Trend (NAEP-LTT), given to students nine, 13, and 17 years of age.  The NAEP-LTT in reading was first administered in 1971.  The second group of results is from the NAEP Main Assessment, which began testing reading achievement in 1992.  It assesses at three different grade levels: fourth, eighth, and twelfth.   The last two tests are international assessments in which the U.S. participates, the Progress in International Reading Literacy Study (PIRLS), which began in 2001, and the Program for International Student Assessment (PISA), first given in 2000.  PIRLS tests fourth graders, and PISA tests 15-year-olds.  In the U.S., 71 percent of students who took PISA in the fall of 2012 were in tenth grade. 

Two findings leap out.  First, the test score gaps between males and females are statistically significant on all eight assessments.  Because the sample sizes of the assessments are quite large, statistical significance does not necessarily mean that the gaps are of practical significance—or even noticeable if one observed several students reading together.  The tests also employ different scales.  The final column in the table expresses the gaps in standard deviation units, a measure that allows for comparing the different scores and estimating their practical meaningfulness.

The second finding is based on the standardized gaps (expressed in SDs).  On both NAEP tests, the gaps are narrower among elementary students and wider among middle and high school students.  That pattern also appears on international assessments.  The gap is twice as large on PISA as on PIRLS.[vi]  A popular explanation for the gender gap involves the different maturation rates of boys and girls.  That theory will be discussed in greater detail below, but at this point in the analysis, let’s simply note that the gender gap appears to grow until early adolescence—age 13 on the LTT-NAEP and grade eight on the NAEP Main.

Should these gaps be considered small or large?  Many analysts consider 10 scale score points on NAEP equal to about a year of learning.  In that light, gaps of five to 10 points appear substantial.  But compared to other test score gaps on NAEP, the gender gap is modest in size.  On the 2012 LTT-NAEP for nine-year-olds, the five point gap between boys and girls is about one-half of the 10 point gap between students living in cities and those living in suburbs.[vii]  The gap between students who are eligible for free and reduced lunch and those who are not is 28 points; between black and white students, it is 23 points; and between English language learners (ELL) and non-ELL students, it is 34 points. 

Table 1-1 only shows the size of the gender gap as gauged by assessments at single points in time.  For determining trends, let’s take a closer look at the LTT-NAEP, since it provides the longest running record of the gender gap.  In Table 1-2, scores are displayed from tests administered since 1971 and given nearest to the starts and ends of decades.  Results from 2008 and 2012 are both shown to provide readers an idea of recent fluctuations.  At all three ages, gender gaps were larger in 1971 than they are today.  The change at age nine is statistically significant, but not at age 13 (p=0.10) or age 17 (p=.07), although they are close.  Slight shrinkage occurred in the 1980s, but the gaps expanded again in the 1990s.  The gap at age 13 actually peaked at 15 scale score points in 1994 (not shown in the table), and the decline since then is statistically significant.  Similarly, the gap at age 17 peaked in 1996 at 15 scale score points, and the decline since then is also statistically significant.  More recently, the gap at age nine began to shrink again in 1999, age 13 began shrinking in the 2000s, and age 17 in 2012.

Table 1-3 decomposes the change figures by male and female performance.  Sara Mead’s point, that the NAEP story is one of both sexes gaining rather than boys falling behind, is even truer today than when she made it in 2006.  When Mead’s analysis was published, the most recent LTT-NAEP data were from 2004.  Up until then, girls had made greater reading gains than boys.  But that situation has reversed.  Boys have now made larger gains over the history of LTT-NAEP, fueled by the gains that they registered from 2004 to 2012.  The score for 17-year-old females in 2012 (291) was identical to their score in 1971.

The United States is not alone in reading’s gender gap.  Its gap of 31 points is not even the largest (see Figure 1-1). On the 2012 PISA, all OECD countries exhibited a gender gap, with females outscoring males by 23 to 62 points on the PISA scale (standard deviation of 94).   On average in the OECD, girls outscored boys by 38 points (rounded to 515 for girls and 478 for boys).  The U.S. gap of 31 points is less than the OECD average.

Finland had the largest gender gap on the 2012 PISA, twice that of the U.S., with females outscoring males by an astonishing 62 points (0.66 SDs).  Finnish girls scored 556, and boys scored 494.  To put this gap in perspective, consider that Finland’s renowned superiority on PISA tests is completely dependent on Finnish girls.  Finland’s boys’ score of 494 is about the same as the international average of 496, and not much above the OECD average for males (478).  The reading performance of Finnish boys is not statistically significantly different from boys in the U.S. (482) or from the average U.S. student, both boys and girls (498). Finnish superiority in reading only exists among females.

There is a hint of a geographical pattern.  Northern European countries tend to have larger gender gaps in reading.  Finland, Sweden, Iceland, and Norway have four of the six largest gaps.  Denmark is the exception with a 31 point gap, below the OECD average.   And two Asian OECD members have small gender gaps.  Japan’s gap of 24 points and South Korea’s gap of 23 are ranked among the bottom four countries. The Nordic tendency toward large gender gaps in reading was noted in a 2002 analysis of the 2000 PISA results.^[viii]  At that time, too, Denmark was the exception.  Because of the larger sample and persistence over time, the Nordic pattern warrants more confidence than the one in the two Asian countries.

Back to Finland.  That’s the headline story here, and it contains a lesson for cautiously interpreting international test scores.  Consider that the 62 point gender gap in Finland is only 14 points smaller than the U.S. black-white gap (76 points) and 21 points larger than the white-Hispanic gap (41 points) on the same test.  Finland’s gender gap illustrates the superficiality of much of the commentary on that country’s PISA performance.  A common procedure in policy analysis is to consider how policies differentially affect diverse social groups.  Think of all the commentators who cite Finland to promote particular policies, whether the policies address teacher recruitment, amount of homework, curriculum standards, the role of play in children’s learning, school accountability, or high stakes assessments.^[ix]  Advocates pound the table while arguing that these policies are obviously beneficial.  “Just look at Finland,” they say.  Have you ever read a warning that even if those policies contribute to Finland’s high PISA scores—which the advocates assume but serious policy scholars know to be unproven—the policies also may be having a negative effect on the 50 percent of Finland’s school population that happens to be male?

One of the solutions put forth for improving boys’ reading scores is to make an effort to boost their enjoyment of reading.  That certainly makes sense, but past scores of national reading and math performance have consistently, and counterintuitively, shown no relationship (or even an inverse one) with enjoyment of the two subjects.  PISA asks students how much they enjoy reading, so let’s now investigate whether fluctuations in PISA scores are at all correlated with how much 15-year-olds say they like to read.

The analysis below employs what is known as a “differences-in-differences” analytical strategy.  In both 2000 and 2009, PISA measured students’ reading ability and asked them several questions about how much they like to read.  An enjoyment index was created from the latter set of questions.^[x]  Females score much higher on this index than boys.  Many commentators believe that girls’ greater enjoyment of reading may be at the root of the gender gap in literacy.

When new international test scores are released, analysts are tempted to just look at variables exhibiting strong correlations with achievement (such as amount of time spent on homework), and embrace them as potential causes of high achievement. But cross-sectional correlations can be deceptive.  The direction of causality cannot be determined, whether it’s doing a lot of homework that leads to high achievement, or simply that good students tend to take classes that assign more homework.  Correlations in cross-sectional data are also vulnerable to unobserved factors that may influence achievement.  For example, if cultural predilections drive a country’s exemplary performance, their influence will be masked or spuriously assigned to other variables unless they are specifically modeled.^[xi]  Class size, between-school tracking, and time spent on learning are all topics on which differences-in-differences has been fruitfully employed to analyze multiple cross-sections of international data.

Another benefit of differences-in-differences is that it measures statistical relationships longitudinally.  Table 1-4 investigates the question: Is the rise and fall of reading enjoyment correlated with changes in reading achievement?  Many believe that if boys liked reading more, their literacy test scores would surely increase.  Table 1-4 does not support that belief.  Data are available for 27 OECD countries, and they are ranked by how much they boosted males’ enjoyment of reading.  The index is set at the student-level with a mean of 0.00 and standard deviation of 1.00.  For the twenty-seven nations in Table 1-4, the mean national change in enjoyment is -.02 with a standard deviation of .09. 

Germany did the best job of raising boys’ enjoyment of reading, with a gain of 0.12 on the index.  German males’ PISA scores also went up—a little more than 10 points (10.33).  France, on the other hand, raised males’ enjoyment of reading nearly as much as Germany (0.11), but French males’ PISA scores declined by 15.26 points.  A bit further down the column, Ireland managed to get boys to enjoy reading a little more (a gain of 0.05) but their reading performance fell a whopping 36.54 points.  Toward the bottom end of the list, Poland’s boys enjoyed reading less in 2009 than in 2000, a decline of 0.14 on the index, but over the same time span, their reading literacy scores increased by more than 14 points (14.29).  Among the countries in which the relationship goes in the expected direction is Finland.  Finnish males’ enjoyment of reading declined (-0.14) as did their PISA scores in reading literacy (-11.73).  Overall, the correlation coefficient for change in enjoyment and change in reading score is -0.01, indicating no relationship between the two.

Christina Hoff Sommers and Richard Whitmire have praised specific countries for first recognizing and then addressing the gender gap in reading.  Recently, Sommers urged the U.S. to “follow the example of the British, Canadians, and Australians.”^[xii]  Whitmire described Australia as “years ahead of the U.S. in pioneering solutions” to the gender gap.  Let’s see how those countries appear in Table 1-4.  England does not have PISA data for the 2000 baseline year, but both Canada and Australia are included.  Canada raised boys’ enjoyment of reading a little bit (0.02) but Canadian males’ scores fell by about 12 points (-11.74).  Australia suffered a decline in boys’ enjoyment of reading (-0.04) and achievement (-16.50).  As promising as these countries’ efforts may have appeared a few years ago, so far at least, they have not borne fruit in raising boys’ reading performance on PISA.

Achievement gaps are tricky because it is possible for the test scores of the two groups being compared to both decline while the gap increases or, conversely, for scores of both to increase while the gap declines.  Table 1-4 only looks at males’ enjoyment of reading and its relationship to achievement.  A separate differences-in-differences analysis was conducted (but not displayed here) to see whether changes in the enjoyment gap—the difference between boys’ and girls’ enjoyment of reading—are related to changes in reading achievement.  They are not (correlation coefficient of 0.08).  National PISA data simply do not support the hypothesis that the superior reading performance of girls is related to the fact that girls enjoy reading more than boys. 

Let’s summarize the main findings of the analysis above. Reading scores for girls exceed those for boys on eight recent assessments of U.S. reading achievement.  The gender gap is larger for middle and high school students than for students in elementary school.  The gap was apparent on the earliest NAEP tests in the 1970s and has shown some signs of narrowing in the past decade.  International tests reveal that the gender gap is worldwide.  Among OECD countries, it even appears among countries known for superior performance on PISA’s reading test.  Finland not only exhibited the largest gender gap in reading on the 2012 PISA, the gap had widened since 2000.  A popular recommendation for boosting boys’ reading performance is finding ways for them to enjoy reading more.  That theory is not supported by PISA data.  Countries that succeeded in raising boys’ enjoyment of reading from 2000 to 2009 were no more likely to improve boys’ reading performance than countries where boys’ enjoyment of reading declined. 

The origins of the gender gap are hotly debated.  The universality of the gap certainly supports the argument that it originates in biological or developmental differences between the two sexes.  It is evident among students of different ages in data collected at different points in time.  It exists across the globe, in countries with different educational systems, different popular cultures, different child rearing practices, and different conceptions of gender roles.  Moreover, the greater prevalence of reading impairment among young boys—a ratio of two or three to one—suggests an endemic difficulty that exists before the influence of schools or culture can take hold.^[xiii] 

But some of the data examined above also argue against the developmental explanation.  The gap has been shrinking on NAEP.  At age nine, it is less than half of what it was forty years ago.  Biology doesn’t change that fast.  Gender gaps in math and science, which were apparent in achievement data for a long time, have all but disappeared, especially once course taking is controlled.  The reading gap also seems to evaporate by adulthood.  On an international assessment of adults conducted in 2012, reading scores for men and women were statistically indistinguishable up to age 35—even in Finland and the United States.  After age 35, men had statistically significantly higher scores in reading, all the way to the oldest group, age 55 and older.  If the gender gap in literacy is indeed shaped by developmental factors, it may be important for our understanding of the phenomenon to scrutinize periods of the life cycle beyond the age of schooling.   

Another astonishing pattern emerged from the study of adult reading.  Participants were asked how often they read a book.  Of avid book readers (those who said they read a book once a week) in the youngest group (age 24 and younger), 59 percent were women and 41 percent were men.  By age 55, avid book readers were even more likely to be women, by a margin of 63 percent to 37 percent.  Two-thirds of respondents who said they never read books were men.  Women remained the more enthusiastic readers even as the test scores of men caught up with those of women and surpassed them.

A few years ago, Ian McEwan, the celebrated English novelist, decided to reduce the size of the library in his London townhouse.  He and his younger son selected thirty novels and took them to a local park.  They offered the books to passers-by.  Women were eager and grateful to take the books, McEwan reports.  Not a single man accepted.  The author’s conclusion? “When women stop reading, the novel will be dead.”^[xiv] 

McEwan might be right, regardless of the origins of the gender gap in reading and the efforts to end it.

Downloads

• Download the report

Part II of the 2015 Brown Center Report on American Education

Over the next several years, policy analysts will evaluate the impact of the Common Core State Standards (CCSS) on U.S. education.  The task promises to be challenging.  The question most analysts will focus on is whether the CCSS is good or bad policy.  This section of the Brown Center Report (BCR) tackles a set of seemingly innocuous questions compared to the hot-button question of whether Common Core is wise or foolish.  The questions all have to do with when Common Core actually started, or more precisely, when the Common Core started having an effect on student learning.  And if it hasn’t yet had an effect, how will we know that CCSS has started to influence student achievement? 

The analysis below probes this issue empirically, hopefully persuading readers that deciding when a policy begins is elemental to evaluating its effects.  The question of a policy’s starting point is not always easy to answer.  Yet the answer has consequences.  You can’t figure out whether a policy worked or not unless you know when it began.^[i] 

The analysis uses surveys of state implementation to model different CCSS starting points for states and produces a second early report card on how CCSS is doing.  The first report card, focusing on math, was presented in last year’s BCR.  The current study updates state implementation ratings that were presented in that report and extends the analysis to achievement in reading.  The goal is not only to estimate CCSS’s early impact, but also to lay out a fair approach for establishing when the Common Core’s impact began—and to do it now before data are generated that either critics or supporters can use to bolster their arguments.  The experience of No Child Left Behind (NCLB) illustrates this necessity.

After the 2008 National Assessment of Educational Progress (NAEP) scores were released, former Secretary of Education Margaret Spellings claimed that the new scores showed “we are on the right track.”^[ii] She pointed out that NAEP gains in the previous decade, 1999-2009, were much larger than in prior decades.  Mark Schneider of the American Institutes of Research (and a former Commissioner of the National Center for Education Statistics [NCES]) reached a different conclusion. He compared NAEP gains from 1996-2003 to 2003-2009 and declared NCLB’s impact disappointing.  “The pre-NCLB gains were greater than the post-NCLB gains.”^[iii]  It is important to highlight that Schneider used the 2003 NAEP scores as the starting point for assessing NCLB.  A report from FairTest on the tenth anniversary of NCLB used the same demarcation for pre- and post-NCLB time frames.^[iv]  FairTest is an advocacy group critical of high stakes testing—and harshly critical of NCLB—but if the 2003 starting point for NAEP is accepted, its conclusion is indisputable, “NAEP score improvement slowed or stopped in both reading and math after NCLB was implemented.” 

Choosing 2003 as NCLB’s starting date is intuitively appealing.  The law was introduced, debated, and passed by Congress in 2001.  President Bush signed NCLB into law on January 8, 2002.  It takes time to implement any law.  The 2003 NAEP is arguably the first chance that the assessment had to register NCLB’s effects. 

Selecting 2003 is consequential, however.  Some of the largest gains in NAEP’s history were registered between 2000 and 2003.  Once 2003 is established as a starting point (or baseline), pre-2003 gains become “pre-NCLB.”  But what if the 2003 NAEP scores were influenced by NCLB? Experiments evaluating the effects of new drugs collect baseline data from subjects before treatment, not after the treatment has begun.   Similarly, evaluating the effects of public policies require that baseline data are not influenced by the policies under evaluation.   

Avoiding such problems is particularly difficult when state or local policies are adopted nationally.  The federal effort to establish a speed limit of 55 miles per hour in the 1970s is a good example.  Several states already had speed limits of 55 mph or lower prior to the federal law’s enactment.  Moreover, a few states lowered speed limits in anticipation of the federal limit while the bill was debated in Congress.  On the day President Nixon signed the bill into law—January 2, 1974—the Associated Press reported that only 29 states would be required to lower speed limits.  Evaluating the effects of the 1974 law with national data but neglecting to adjust for what states were already doing would obviously yield tainted baseline data.

There are comparable reasons for questioning 2003 as a good baseline for evaluating NCLB’s effects.  The key components of NCLB’s accountability provisions—testing students, publicizing the results, and holding schools accountable for results—were already in place in nearly half the states.  In some states they had been in place for several years.  The 1999 iteration of Quality Counts, Education Week’s annual report on state-level efforts to improve public education, entitled Rewarding Results, Punishing Failure, was devoted to state accountability systems and the assessments underpinning them. Testing and accountability are especially important because they have drawn fire from critics of NCLB, a law that wasn’t passed until years later.

The Congressional debate of NCLB legislation took all of 2001, allowing states to pass anticipatory policies.  Derek Neal and Diane Whitmore Schanzenbach reported that “with the passage of NCLB lurking on the horizon,” Illinois placed hundreds of schools on a watch list and declared that future state testing would be high stakes.^[v] In the summer and fall of 2002, with NCLB now the law of the land, state after state released lists of schools falling short of NCLB’s requirements.  Then the 2002-2003 school year began, during which the 2003 NAEP was administered.  Using 2003 as a NAEP baseline assumes that none of these activities—previous accountability systems, public lists of schools in need of improvement, anticipatory policy shifts—influenced achievement.  That is unlikely.^[vi]

Unlike NCLB, there was no “pre-CCSS” state version of Common Core.  States vary in how quickly and aggressively they have implemented CCSS.  For the BCR analyses, two indexes were constructed to model CCSS implementation.  They are based on surveys of state education agencies and named for the two years that the surveys were conducted.  The 2011 survey reported the number of programs (e.g., professional development, new materials) on which states reported spending federal funds to implement CCSS.  Strong implementers spent money on more activities.  The 2011 index was used to investigate eighth grade math achievement in the 2014 BCR.  A new implementation index was created for this year’s study of reading achievement.  The 2013 index is based on a survey asking states when they planned to complete full implementation of CCSS in classrooms.  Strong states aimed for full implementation by 2012-2013 or earlier.      

Fourth grade NAEP reading scores serve as the achievement measure.  Why fourth grade and not eighth?  Reading instruction is a key activity of elementary classrooms but by eighth grade has all but disappeared.  What remains of “reading” as an independent subject, which has typically morphed into the study of literature, is subsumed under the English-Language Arts curriculum, a catchall term that also includes writing, vocabulary, listening, and public speaking.  Most students in fourth grade are in self-contained classes; they receive instruction in all subjects from one teacher.  The impact of CCSS on reading instruction—the recommendation that non-fiction take a larger role in reading materials is a good example—will be concentrated in the activities of a single teacher in elementary schools. The burden for meeting CCSS’s press for non-fiction, on the other hand, is expected to be shared by all middle and high school teachers.^[vii] 

Table 2-1 displays NAEP gains using the 2011 implementation index.  The four year period between 2009 and 2013 is broken down into two parts: 2009-2011 and 2011-2013.  Nineteen states are categorized as “strong” implementers of CCSS on the 2011 index, and from 2009-2013, they outscored the four states that did not adopt CCSS by a little more than one scale score point (0.87 vs. -0.24 for a 1.11 difference).  The non-adopters are the logical control group for CCSS, but with only four states in that category—Alaska, Nebraska, Texas, and Virginia—it is sensitive to big changes in one or two states.  Alaska and Texas both experienced a decline in fourth grade reading scores from 2009-2013.

The 1.11 point advantage in reading gains for strong CCSS implementers is similar to the 1.27 point advantage reported last year for eighth grade math.  Both are small.  The reading difference in favor of CCSS is equal to approximately 0.03 standard deviations of the 2009 baseline reading score.  Also note that the differences were greater in 2009-2011 than in 2011-2013 and that the “medium” implementers performed as well as or better than the strong implementers over the entire four year period (gain of 0.99).

Table 2-2 displays calculations using the 2013 implementation index.  Twelve states are rated as strong CCSS implementers, seven fewer than on the 2011 index.^[viii]  Data for the non-adopters are the same as in the previous table.  In 2009-2013, the strong implementers gained 1.27 NAEP points compared to -0.24 among the non-adopters, a difference of 1.51 points.  The thirty-four states rated as medium implementers gained 0.82.  The strong implementers on this index are states that reported full implementation of CCSS-ELA by 2013.  Their larger gain in 2011-2013 (1.08 points) distinguishes them from the strong implementers in the previous table.  The overall advantage of 1.51 points over non-adopters represents about 0.04 standard deviations of the 2009 NAEP reading score, not a difference with real world significance.  Taken together, the 2011 and 2013 indexes estimate that NAEP reading gains from 2009-2013 were one to one and one-half scale score points larger in the strong CCSS implementation states compared to the states that did not adopt CCSS.

As noted above, the 2013 implementation index is based on when states scheduled full implementation of CCSS in classrooms.  Other than reading achievement, does the index seem to reflect changes in any other classroom variable believed to be related to CCSS implementation?  If the answer is “yes,” that would bolster confidence that the index is measuring changes related to CCSS implementation. 

Let’s examine the types of literature that students encounter during instruction.  Perhaps the most controversial recommendation in the CCSS-ELA standards is the call for teachers to shift the content of reading materials away from stories and other fictional forms of literature in favor of more non-fiction.  NAEP asks fourth grade teachers the extent to which they teach fiction and non-fiction over the course of the school year (see Figure 2-1). 

Historically, fiction dominates fourth grade reading instruction.  It still does.  The percentage of teachers reporting that they teach fiction to a “large extent” exceeded the percentage answering “large extent” for non-fiction by 23 points in 2009 and 25 points in 2011.  In 2013, the difference narrowed to only 15 percentage points, primarily because of non-fiction’s increased use.  Fiction still dominated in 2013, but not by as much as in 2009.

The differences reported in Table 2-3 are national indicators of fiction’s declining prominence in fourth grade reading instruction.  What about the states?  We know that they were involved to varying degrees with the implementation of Common Core from 2009-2013.  Is there evidence that fiction’s prominence was more likely to weaken in states most aggressively pursuing CCSS implementation? 

Table 2-3 displays the data tackling that question.  Fourth grade teachers in strong implementation states decisively favored the use of fiction over non-fiction in 2009 and 2011.  But the prominence of fiction in those states experienced a large decline in 2013 (-12.4 percentage points).  The decline for the entire four year period, 2009-2013, was larger in the strong implementation states (-10.8) than in the medium implementation (-7.5) or non-adoption states (-9.8).  

This section of the Brown Center Report analyzed NAEP data and two indexes of CCSS implementation, one based on data collected in 2011, the second from data collected in 2013.  NAEP scores for 2009-2013 were examined.  Fourth grade reading scores improved by 1.11 scale score points in states with strong implementation of CCSS compared to states that did not adopt CCSS.  A similar comparison in last year’s BCR found a 1.27 point difference on NAEP’s eighth grade math test, also in favor of states with strong implementation of CCSS.  These differences, although certainly encouraging to CCSS supporters, are quite small, amounting to (at most) 0.04 standard deviations (SD) on the NAEP scale.  A threshold of 0.20 SD—five times larger—is often invoked as the minimum size for a test score change to be regarded as noticeable.  The current study’s findings are also merely statistical associations and cannot be used to make causal claims.  Perhaps other factors are driving test score changes, unmeasured by NAEP or the other sources of data analyzed here. 

The analysis also found that fourth grade teachers in strong implementation states are more likely to be shifting reading instruction from fiction to non-fiction texts.  That trend should be monitored closely to see if it continues.  Other events to keep an eye on as the Common Core unfolds include the following:

1.  The 2015 NAEP scores, typically released in the late fall, will be important for the Common Core.  In most states, the first CCSS-aligned state tests will be given in the spring of 2015.  Based on the earlier experiences of Kentucky and New York, results are expected to be disappointing.  Common Core supporters can respond by explaining that assessments given for the first time often produce disappointing results.  They will also claim that the tests are more rigorous than previous state assessments.  But it will be difficult to explain stagnant or falling NAEP scores in an era when implementing CCSS commands so much attention.   

2.  Assessment will become an important implementation variable in 2015 and subsequent years.  For analysts, the strategy employed here, modeling different indicators based on information collected at different stages of implementation, should become even more useful.  Some states are planning to use Smarter Balanced Assessments, others are using the Partnership for Assessment of Readiness for College and Careers (PARCC), and still others are using their own homegrown tests.   To capture variation among the states on this important dimension of implementation, analysts will need to use indicators that are up-to-date.

3.  The politics of Common Core injects a dynamic element into implementation.  The status of implementation is constantly changing.  States may choose to suspend, to delay, or to abandon CCSS.  That will require analysts to regularly re-configure which states are considered “in” Common Core and which states are “out.”  To further complicate matters, states may be “in” some years and “out” in others.

A final word.  When the 2014 BCR was released, many CCSS supporters commented that it is too early to tell the effects of Common Core.  The point that states may need more time operating under CCSS to realize its full effects certainly has merit.  But that does not discount everything states have done so far—including professional development, purchasing new textbooks and other instructional materials, designing new assessments, buying and installing computer systems, and conducting hearings and public outreach—as part of implementing the standards.  Some states are in their fifth year of implementation.  It could be that states need more time, but innovations can also produce their biggest “pop” earlier in implementation rather than later.  Kentucky was one of the earliest states to adopt and implement CCSS.  That state’s NAEP fourth grade reading score declined in both 2009-2011 and 2011-2013.  The optimism of CCSS supporters is understandable, but a one and a half point NAEP gain might be as good as it gets for CCSS.

Downloads

• Download the report

This week marks the release of the 2015 Brown Center Report on American Education, the fourteenth issue of the series.  One of the three studies in the report, “Girls, Boys, and Reading,” examines the gender gap in reading.  Girls consistently outscore boys on reading assessments.  They have for a long time.  A 1942 study in Iowa discovered that girls were superior to boys on tests of reading comprehension, vocabulary, and basic language skills.^[i]  Girls have outscored boys on the National Assessment of Educational Progress (NAEP) reading assessments since the first NAEP was administered in 1971. 

I hope you’ll read the full study—and the other studies in the report—but allow me to summarize the main findings of the gender gap study here.

Eight assessments generate valid estimates of U.S. national reading performance: the Main NAEP, given at three grades (fourth, eighth, and 12^th grades); the NAEP Long Term Trend (NAEP-LTT), given at three ages (ages nine, 13, and 17); the Progress in International Reading Literacy Study (PIRLS), an international assessment given at fourth grade; and the Program for International Student Assessment (PISA), an international assessment given to 15-year-olds.  Females outscore males on the most recent administration of all eight tests.  And the gaps are statistically significant.  Expressed in standard deviation units, they range from 0.13 on the NAEP-LTT at age nine to 0.34 on the PISA at age 15.

The gaps are shrinking.  At age nine, the gap on the NAEP-LTT declined from 13 scale score points in 1971 to five points in 2012.  During the same time period, the gap at age 13 shrank from 11 points to eight points, and at age 17, from 12 points to eight points.  Only the decline at age nine is statistically significant, but at ages 13 and 17, declines since the gaps peaked in the 1990s are also statistically significant.  At all three ages, gaps are shrinking because of males making larger gains on NAEP than females.  In 2012, seventeen-year-old females scored the same on the NAEP reading test as they did in 1971.  Otherwise, males and females of all ages registered gains on the NAEP reading test from 1971-2012, with males’ gains outpacing those of females.

The gap is worldwide.  On the 2012 PISA, 15-year-old females outperformed males in all sixty-five participating countries.  Surprisingly, Finland, a nation known for both equity and excellence because of its performance on PISA, evidenced the widest gap.  Girls scored 556 and boys scored 494, producing an astonishing gap of 62 points (about 0.66 standard deviations—or more than one and a half years of schooling).   Finland also had one of the world’s largest gender gaps on the 2000 PISA, and since then it has widened.  Both girls’ and boys’ reading scores declined, but boys’ declined more (26 points vs. 16 points).  To put the 2012 scores in perspective, consider that the OECD average on the reading test is 496.  Finland’s strong showing on PISA is completely dependent on the superior performance of its young women.

The gap seems to disappear by adulthood.  Tests of adult reading ability show no U.S. gender gap in reading by 25 years of age.  Scores even tilt toward men in later years. 

The words “seems to disappear” are used on purpose.  One must be careful with cross-sectional data not to assume that differences across age groups indicate an age-based trend.  A recent Gallup poll, for example, asked several different age groups how optimistic they were about finding jobs as adults.  Optimism fell from 68% in grade five to 48% in grade 12.  The authors concluded that “optimism about future job pursuits declines over time.”  The data do not support that conclusion.  The data were collected at a single point in time and cannot speak to what optimism may have been before or after that point.  Perhaps today’s 12^th graders were even more pessimistic several years ago when they were in fifth grade.  Perhaps the 12^th-graders are old enough to remember when unemployment spiked during the Great Recession and the fifth-graders are not.   Perhaps 12^th-graders are simply savvier about job prospects and the pitfalls of seeking employment, topics on which fifth-graders are basically clueless.

At least with the data cited above we can track measures of the same cohorts’ gender gap in reading over time.  By analyzing multiple cross-sections—data collected at several different points in time—we can look at real change.  Those cohorts of nine-year-olds in the 1970s, 1980s, and 1990s, are—respectively—today in their 50s, 40s, and 30s.  Girls were better readers than boys when these cohorts were children, but as grown ups, women are not appreciably better readers than men.

Care must be taken nevertheless in drawing firm conclusions.  There exists what are known as cohort effects that can bias measurements.  I mentioned the Great Recession.   Experiencing great historical cataclysms, especially war or economic chaos, may bias a particular cohort’s responses to survey questions or even its performance on tests.  American generations who experienced the Great Depression, World War II, and the Vietnam War—and more recently, the digital revolution, the Great Recession, and the Iraq War—lived through events that uniquely shape their outlook on many aspects of life. 

The gender gap is large, worldwide, and persistent through the K-12 years. What should be done about it?  Maybe nothing.  As just noted, the gap seems to dissipate by adulthood.  Moreover, crafting an effective remedy for the gender gap is made more difficult because we don’t definitely know its cause. Enjoyment of reading is a good example.  Many commentators argue that schools should make a concerted effort to get boys to enjoy reading more.  Enjoyment of reading is statistically correlated with reading performance, and the hope is that making reading more enjoyable would get boys to read more, thereby raising reading skills.

It makes sense, but I’m skeptical.  The fact that better readers enjoy reading more than poor readers—and that the relationship stands up even after boatloads of covariates are poured into a regression equation—is unpersuasive evidence of causality.  As I stated earlier, PISA produces data collected at a single point in time.  It isn’t designed to test causal theories.  Reverse causality is a profound problem.  Getting kids to enjoy reading more may in fact boost reading ability.  But the causal relationship might be flowing in the opposite direction, with enhanced skill leading to enjoyment.   The correlation could simply be indicating that people enjoy activities that they’re good at—a relationship that probably exists in sports, music, and many human endeavors, including reading.

A key question for policymakers is whether boosting boys’ enjoyment of reading would help make boys better readers.  I investigate by analyzing national changes in PISA reading scores from 2000, when the test was first given, to 2102.  PISA creates an Index of Reading Enjoyment based on several responses to a student questionnaire.  Enjoyment of reading has increased among males in some countries and decreased in others.  Is there any relationship between changes in boys’ enjoyment and changes in PISA reading scores? 

There is not.  The correlation coefficient for the two phenomena is -0.01.  Nations such as Germany raised boys’ enjoyment of reading and increased their reading scores by about 10 points on the PISA scale.  France, on the other hand, also raised boys’ enjoyment of reading, but French males’ reading scores declined by 15 points.  Ireland increased how much boys enjoy reading by a little bit but the boys’ scores fell a whopping 37 points. Poland’s males actually enjoyed reading less in 2012 than in 2000, but their scores went up more than 14 points.  No relationship.

How should policymakers proceed?  Large, cross-sectional assessments are good for measuring academic performance at one point in time.  They are useful for generating hypotheses based on observed relationships, but they are not designed to confirm or reject causality.  To do that, randomized control trials should be conducted of programs purporting to boost reading enjoyment.  Also, consider that it ultimately may not matter whether enjoying reading leads to more proficient readers.  Enjoyment of reading may be an end worthy of attainment irrespective of its relationship to achievement.  In that case, RCTs should carefully evaluate the impact of interventions on both enjoyment of reading and reading achievement, whether the two are related or not.  

Event Information

And more from the Brown Center Report on American Education

Girls outscore boys on practically every reading test given to a large population. And they have for a long time. A 1942 Iowa study found girls performing better than boys on tests of reading comprehension, vocabulary, and basic language skills, and girls have outscored boys on every reading test ever given by the National Assessment of Educational Progress (NAEP). This gap is not confined to the U.S. Reading tests administered as part of the Progress in International Reading Literacy Study (PIRLS) and the Program for International Student Assessment (PISA) reveal that the gender gap is a worldwide phenomenon.

On March 26, join Brown Center experts Tom Loveless and Matthew Chingos as they discuss the latest Brown Center Report on American Education, which examines this phenomenon. Hear what Loveless's analysis revealed about where the gender gap stands today and how it's trended over the past several decades - in the U.S. and around the world.

Tune in below or via Spreecast where you can submit questions. 

Spreecast is the social video platform that connects people.
Check out Girls, Boys, and Reading on Spreecast.

Last week, CNN ran a back-to-school story on homework with the headline, “Kids Have Three Times Too Much Homework, Study Finds; What’s the Cost?” Homework is an important topic, especially for parents, but unfortunately, CNN’s story misleads rather than informs. The headline suggests American parents should be alarmed because their kids have too much homework. Should they? No, CNN has ignored the best evidence on that question, which suggests the opposite. The story relies on the results of one recent study of homework—a study that is limited in what it can tell us, mostly because of its research design. But CNN even gets its main findings wrong. The study suggests most students have too little homework, not too much.

The study that piqued CNN’s interest was conducted during four months (two in the spring and two in the fall) in Providence, Rhode Island. About 1,200 parents completed a survey about their children’s homework while waiting in 27 pediatricians’ offices. Is the sample representative of all parents in the U.S.? Probably not. Certainly CNN should have been a bit leery of portraying the results of a survey conducted in a single American city—any city—as evidence applying to a broader audience. More importantly, viewers are never told of the study’s significant limitations: that the data come from a survey conducted in only one city—in pediatricians’ offices by a self-selected sample of respondents.

The survey’s sampling design is a huge problem. Because the sample is non-random there is no way of knowing if the results can be extrapolated to a larger population—even to families in Providence itself. Close to a third of respondents chose to complete the survey in Spanish. Enrollment in English Language programs in the Providence district comprises about 22 percent of students. About one-fourth (26 percent) of survey respondents reported having one child in the family. According to the 2010 Census, the proportion of families nationwide with one child is much higher, at 43 percent.^[i] The survey is skewed towards large, Spanish-speaking families. Their experience with homework could be unique, especially if young children in these families are learning English for the first time at school.

The survey was completed by parents who probably had a sick child as they were waiting to see a pediatrician. That’s a stressful setting. The response rate to the survey is not reported, so we don’t know how many parents visiting those offices chose not to fill out the survey. If the typical pediatrician sees 100 unique patients per month, in a four month span the survey may have been offered to more than ten thousand parents in the 27 offices. The survey respondents, then, would be a tiny slice, 10 to 15 percent, of those eligible to respond. We also don’t know the public-private school break out of the respondents, or how many were sending their children to charter schools. It would be interesting to see how many parents willingly send their children to schools with a heavy homework load.

I wish the CNN team responsible for this story had run the data by some of CNN’s political pollsters. Alarm bells surely would have gone off. The hazards of accepting a self-selected, demographically-skewed survey sample as representative of the general population are well known. Modern political polling—and its reliance on random samples—grew from an infamous mishap in 1936. A popular national magazine, the Literary Digest, distributed 10 million post cards for its readers to return as “ballots” indicating who they would vote for in the 1936 race for president. More than two million post cards were returned! A week before the election, the magazine confidently predicted that Alf Landon, the Republican challenger from Kansas, would defeat Franklin Roosevelt, the Democratic incumbent, by a huge margin: 57 percent to 43 percent. In fact, when the real election was held, the opposite occurred: Roosevelt won more than 60% of the popular vote and defeated Landon in a landslide. Pollsters learned that self-selected samples should be viewed warily. The magazine’s readership was disproportionately Republican to begin with, and sometimes disgruntled subjects are more likely to respond to a survey, no matter the topic, than the satisfied.

Here’s a very simple question: In its next poll on the 2016 presidential race, would CNN report the results of a survey of self-selected respondents in 27 pediatricians’ offices in Providence, Rhode Island as representative of national sentiment? Of course not. Then, please, CNN, don’t do so with education topics.

Let’s set aside methodological concerns and turn to CNN’s characterization of the survey’s findings. Did the study really show that most kids have too much homework? No, the headline that “Kids Have Three Times Too Much Homework” is not even an accurate description of the study’s findings. CNN’s on air coverage extended the misinformation. The online video of the coverage is tagged “Study: Your Kids Are Doing Too Much Homework.” The first caption that viewers see is “Study Says Kids Getting Way Too Much Homework.” All of these statements are misleading.

In the published version of the Providence study, the researchers plotted the average amount of time spent on homework by students’ grade.^[ii] They then compared those averages to a “10 minutes per-grade” guideline that serves as an indicator of the “right” amount of homework. I have attempted to replicate the data here in table form (they were originally reported in a line graph) to make that comparison easier.^[iii]

Contrary to CNN’s reporting, the data suggest—based on the ten minute per-grade rule—that most kids in this study have too little homework, not too much. Beginning in fourth grade, the average time spent on homework falls short of the recommended amount—a gap of only four minutes in fourth grade that steadily widens in later grades.

A more accurate headline would have been, “Study Shows Kids in Nine out of 13 Grades Have Too Little Homework.” It appears high school students (grades 9-12) spend only about half the recommended time on homework. Two hours of nightly homework is recommended for 12^th graders. They are, after all, only a year away from college. But according to the Providence survey, their homework load is less than an hour.

So how in the world did CNN come up with the headline “Kids Have Three Times Too Much Homework?” By focusing on grades K-3 and ignoring all other grades. Here’s the reporting:

The study, published Wednesday in The American Journal of Family Therapy, found students in the early elementary school years are getting significantly more homework than is recommended by education leaders, in some cases nearly three times as much homework as is recommended.

The standard, endorsed by the National Education Association and the National Parent-Teacher Association, is the so-called "10-minute rule"— 10 minutes per-grade level per-night. That translates into 10 minutes of homework in the first grade, 20 minutes in the second grade, all the way up to 120 minutes for senior year of high school. The NEA and the National PTA do not endorse homework for kindergarten.

In the study involving questionnaires filled out by more than 1,100 English and Spanish speaking parents of children in kindergarten through grade 12, researchers found children in the first grade had up to three times the homework load recommended by the NEA and the National PTA.

Parents reported first-graders were spending 28 minutes on homework each night versus the recommended 10 minutes. For second-graders, the homework time was nearly 29 minutes, as opposed to the 20 minutes recommended.

And kindergartners, their parents said, spent 25 minutes a night on after-school assignments, according to the study…

CNN focused on the four grades, K-3, in which homework exceeds the ten-minute rule. They ignored more than two-thirds of the grades. Even with this focus, a more accurate headline would have been, “Study Suggests First Graders in Providence, RI Have Three Times Too Much Homework.”

Homework is a controversial topic. People hold differing points of view as to whether there is too much, too little, or just the right amount of homework. That makes it vitally important that the media give accurate information on the empirical dimensions to the debate.  The amount of homework kids should have is subject to debate. But the amount of homework kids actually have is an empirical question. We can debate whether it’s too hot outside, but the actual temperature should be a matter of measurement, not debate. It’s impossible to think of a rational debate that can possibly ensue on the homework issue without knowing the empirical status quo in regards to time. Imagine someone beginning a debate by saying, “I am arguing that kids have too much [substitute “too little” here for the pro-homework side] homework but I must admit that I have no idea how much they currently have.”

Data from the National Assessment of Educational Progress (NAEP) provide the best evidence we have on the amount of homework that kids have. NAEP’s sampling design allows us to make inferences about national trends, and the Long-Term Trend (LTT) NAEP offers data on homework since 1984. The latest LTT NAEP results (2012) indicate that the vast majority of nine-year-olds (83 percent) have less than an hour of homework each night. There has been an apparent uptick in the homework load, however, as 35 percent reported no homework in 1984, and only 22 percent reported no homework in 2012. MET Life also periodically surveys a representative sample of students, parents, and teachers on the homework issue. In the 2007 results, a majority of parents (52 percent) of elementary grade students (grades 3-6 in the MET survey) estimated their children had 30 minutes or less of homework.

The MET Life survey found that parents have an overwhelmingly positive view of the amount of homework their children are assigned. Nine out of ten parents responded that homework offers the opportunity to talk and spend time with their children, and most do not see homework as interfering with family time or as a major source of familial stress. Minority parents, in particular, reported believing homework is beneficial for students’ success at school and in the future.^[iv]

That said, just as there were indeed Alf Landon voters in 1936, there are indeed children for whom homework is a struggle. Some bring home more than they can finish in a reasonable amount of time. A complication for researchers of elementary age children is that the same students who have difficulty completing homework may have other challenges—difficulties with reading, low achievement, and poor grades in school.[v] Parents who question the value of homework often have a host of complaints about their child’s school. It is difficult for researchers to untangle all of these factors and determine, in the instances where there are tensions, whether homework is the real cause. To their credit, the researchers who conducted the Providence study are aware of these constraints and present a number of hypotheses warranting further study with a research design supporting causal inferencing. That’s the value of this research, not CNN’s misleading reporting of the findings.