Editor's Note: The "Scaling up in Agriculture, Rural Development and Nutrition" publication is a series of 20 briefs published by the International Food Policy Research Institute. To read the full publication, click here.

Taking successful development interventions to scale is critical if the world is to achieve the Millennium Development Goals and make essential gains in the fight for improved agricultural productivity, rural incomes, and nutrition. How to support scaling up in these three areas, however, is a major challenge. This collection of policy briefs is designed to contribute to a better understanding of the experience to date and the lessons for the future.

Scaling up means expanding, replicating, adapting, and sustaining successful policies, programs, or projects to reach a greater number of people; it is part of a broader process of innovation and learning. A new idea, model, or approach is typically embodied in a pilot project of limited impact; with monitoring and evaluation, the knowledge acquired from the pilot experience can be used to scale up the model to create larger impacts. The process generally occurs in an iterative and interactive cycle, as the experience from scaling up feeds back into new ideas and learning.

The authors of the 20 policy briefs included here explore the experience of scaling up successful interventions in agriculture, rural development, and nutrition under five broad headings: (1) the role of rural community engagement, (2) the importance of value chains, (3) the intricacies of scaling up nutrition interventions, (4) the lessons learned from institutional approaches, and (5) the experience of international aid donors.

There is no blueprint for when and how to take an intervention to scale, but the examples and experiences described in this series of policy briefs offer important insights into how to address the key global issues of agricultural productivity, food insecurity, and rural poverty.

Editor's Note: On Thursday Brookings Visiting Fellow, Amb. Norm Eisen addressed the Italian Parliament to discuss ethics in government, highlighting efforts in the US to improve transparency and accountability. In the speech, Amb. Eisen argues that while ethics reform can be difficult, it is an absolutely essentially part of any democratic system.

As Prepared For Delivery

Signora Presidente Boldrini, Madam President Brasseur, honorevoli Parlamentari, fellow panelists and distinguished guests, buon pomerigo. Thanks for inviting me to address the urgent subject of ethical standards in political life. It is an honor to be here in the Italian Chamber of Deputies, in this beautiful and ancient city, to which we Americans owe so many elements of our system of government. And in my town of Washington, DC we also borrowed a lot of your architecture, so we owe you for that as well.

In exchange for all that, as a small form of repayment, I would like to offer some perspectives from the U.S. as you consider the adoption of a code of ethical conduct for the Italian Parliament.

Since we are in the Chamber of Deputies, the equivalent of our U.S. Congress’ House of Representatives, I will start with best practices in that body, based on years of my professional life—too many—spent addressing alleged violations of its codes of conduct, including as a defense lawyer and later as the co-founder of a government watchdog group.

And I think there are also important lessons to be drawn from the new, innovative code of conduct for White House officials we established while I served as the "Ethics Czar" of President Obama. At his direction I helped write that new code, the Obama "ethics pledge," and although I am biased I think it has been effective so far, knock wood, there have been no major White House scandals. So I will talk about the lessons of that code of conduct a bit as well. My reflections are those of a friend and partner nation with plenty of challenges of our own. So I approach the issue with genuine humility in sharing our successes and failures.

I. Government ethics while standing on one leg

So—what is our U.S. view of best practices for the contents of government codes of ethics? In the U.S. and dare I say internationally, there is a pretty well developed set of best practices. In our House of Representatives, for example, our equivalent of this Chamber of Deputies, Rule XXIII is the Code of Official Conduct. It provides rules in four core areas;

• one, for regulating conflicts of interest, that is, situations where personal interests or financial holdings may conflict with official parliamentary duties;
• two, for gifts, particularly those from lobbyists and other persons interested in parliamentary decisions;
• three, for outside employment of parliamentarians before, during and after government service, particularly with lobbyists, a situation which we call in the US the revolving door; and
• four, for parliamentarians’ proper use of official resources, that is, hiring, staff, budget, travel and such.

There is much more detail in our code of conduct, and a few other rules as well, but those four items—conflicts, gifts, employment and resources—are the key. These same four key areas are also at the center of our codes of conduct for employees of our executive branch, as codified in our statutes and regulations, as well as in the Obama ethics pledge.

I emphasize these four key items because, having helped draft one code of conduct, and having often delved into many other codes, I sometimes find that I lose the forest for the trees when working with these codes, that the priorities at least for me sometimes get lost in the detail. So I try to keep the core always in mind, though I should add that the content of any such code must of course be particularized for the circumstances of particular government bodies and jurisdictions. Thus our U.S. House code is five pages long, elaborating on those four core items, and the House Ethics Manual of official guidance for the code is 456 pages long. Our Obama ethics pledge we got onto one page, we were proud of that. And we made everyone read and sign that page. To be fair, we could do that because we built on and added to other rules which already existed, and we did have several pages of definitions and references attached to the pledge.

II. Enforcement and transparency

But a good code is only the beginning. In our U.S. experience, just as important as the code, maybe even more important, is its enforcement. And here is where I want to share some lessons drawn from U.S. challenges in recent years, and how we responded. I am going to add two more items to our check list: enforcement and transparency.

Candidly, even with our parliamentary code of conduct in the U.S., our enforcement has sometime lagged. That is in part because under our Constitution, the ultimate enforcers are the parliamentarians themselves, and so they can at times be understandably reluctant to sanction their colleagues and friends. It's human nature.

For example, from about 1998 to 2004, there was a seven-year truce in filing complaints in our House of Representatives. The government watchdog organization I co-founded helped end that in 2004 by writing a complaint together with a brave but lonely member of Congress who was willing to file it with the House Ethics Committee. The resulting investigation resulted in the discipline of the member investigated, and ultimately helped lead to his party losing majority control of the body.

Out of all of that came a new enforcement tool in 2008, in our House of Representatives, that I strongly recommend to you: the creation of a new, independent entity, the Office of Congressional Ethics (OCE). This is a nonpartisan fact-finding body that investigates allegations from any source, including whistleblowers who might otherwise be afraid to step forward. OCE then recommends action to the parliamentarians who constitutionally maintain the ultimate decision-making power. Most importantly, the OCE referrals become public, allowing press, civil society and voter accountability. As a result, I believe, since the creation of the OCE for our House in 2008, there have been a significantly higher number of meritorious investigations there than in our Senate, which does not have a comparable body. The total is about 46 OCE referrals and about 20 House disciplinary actions versus just four letters of admonition by our Senate in that period. To be fair the Senate is a smaller body—but not that much smaller!

That last aspect of OCE enforcement—transparency, and the accountability it brings from media, NGOs and the public—is the sixth and final point I want to emphasize. In our U.S. parliamentary ethics system we have many transparency mechanisms: asset disclosures that our parliamentarians file, disclosures that lobbyists must make about their activities, information in campaign finance filings, and more.

To explain the value of transparency, I would like to close by turning to one of our Obama White House ethics transparency innovations. Starting in 2009, we for the first time put on the Internet virtually all visitor records of those coming to the White House. It used to be that just to get a handful of these records you had to file litigation and wait for years to know who was coming to the White House, who they were meeting with and what the subject of the meeting was. Now millions of Obama White House visitor records are online, each with a dozen or so basic categories of information: the name of the visitor, the person visited, the subject of the meeting and so on.

Why is that important? I began by referencing the Obama White House's record in avoiding major scandal. I think there are a number of reasons for that, including the President's own integrity and the new code of conduct we put into place. But an important part of that success story has also been the fact that records of White House meetings go on the Internet for everyone to see. That transparency brings accountability from the press, civil society and the public. That transparency and accountability has in turn powerfully reinforced the code of conduct: it has discouraged people from having meetings they shouldn't have, and if you don't have the meeting, you can't get in trouble for it.

So the U.S. view in one sentence: regulate conflicts, gifts, employment, and resource use, with strong enforcement and above all transparency. Thanks again for inviting me to share the U.S. perspective. Grazie!

I this week had the pleasure of doing a podcast debate with my Brookings colleague Jonathan Rauch on the question of whether we need stronger machines and weaker transparency in American government, or the opposite. Guess which side I took! This has been a long-running water cooler and cafeteria discussion between Jon and myself since I arrived at Brookings almost a year ago. While we find some areas of agreement in the podcast (more than you might think),I remain unconvinced by the so-called “political realist” school that Jonathan is a leader of. As I have previously written and blogged (here, here and here), I think the realists are fantasists, disconnected from the actual reality of politics, including its risks. We need more transparency, not less to deal with, for example, things like corruption risk, particularly in the post-Citizens United era. Indeed, that decision itself embraces the value of a vigorous transparency regime when other safeguards are relaxed. My belief is that Washington works both more efficiently and more ethically under the scrutinizing gaze of the American media, ngo's and public. As former White House ethics czar, I often facilitated administration openness efforts, including as a means of accountability, for example helping put the White House visitor logs online.

Jon and my lively debate covers not only issues of transparency itself but also applies them to other current topics—the Affordable Care Act, Trade Promotion Authority, and much more. The debate was silently moderated by our colleague Ben Wittes as part of his “Chess Clock Debates” series. With only ten minutes on the chess clock each to make our points, it was a concise discussion that hit the fundamentals briskly. Thanks to Ben for inviting us and giving us a public forum to discuss this critical policy issue.

I was honored today to speak at the kick off meeting of the new ReFormers Caucus. This group of over 100 former members of the U.S. Senate, the House, and governors of both parties, has come together to fight for meaningful campaign finance reform. In the bipartisan spirit of the caucus, I shared speaking duties with Professor Richard Painter, who was the Bush administration ethics czar and my predecessor before I had a similar role in the Obama White House. 

As I told the distinguished audience of ReFormers (get the pun?) gathered over lunch on Capitol Hill, I wish they had existed when in my Obama administration role I was working for the passage of the Disclose Act. That bill would have brought true transparency to the post-Citizens United campaign finance system, yet it failed by just one vote in Congress.  But it is not too late for Americans, working together, to secure enhanced transparency and other campaign finance changes that are desperately needed.  Momentum is building, with increasing levels of public outrage, as reflected in state and local referenda passing in Maine, Seattle and San Francisco just this week, and much more to come at the federal, state and local level.

Event Information

As the sixth anniversary of Citizens United v. FEC approaches on January 21, both experts and ordinary citizens believe the United States is confronting a campaign finance crisis. Citizens United and related court cases have unleashed a flood of dark money that many believe could drown our democracy. It is estimated that over $5 billion will be spent on the 2016 presidential race—more than 3 times the amount spent in 2008 (already the most expensive election cycle in history). A comprehensive poll conducted by the New York Times and CBS News in the spring of 2015 showed that 84 percent of adults—including 90 percent of Democrats and 80 percent of Republicans—believe that money has too much influence in American political campaigns. Even the richest Americans agreed: 85 percent of adults making $100,000 or more share that same belief.

There has been much handwringing about this state of affairs. But there has been too little public attention paid to finding solutions. On the sixth anniversary of Citizens United, the Governance Studies program at Brookings hosted current and former government officials, lobbyists, donors, advocates, and other experts to discuss how to resolve the campaign finance crisis. They focused on innovative reform efforts at the federal, state, and local levels which offer the hope of addressing the problem of big money in politics.

Panelists will included:

Cheri Beasley, Associate Justice, North Carolina Supreme Court
Daniel Berger, Partner, Berger & Montague, P.C.
John Bonifaz, Co-Founder and President, Free Speech for People
Norman L. Eisen, U.S. Ambassador to the Czech Republic (2011-2014); Special Assistant and Special Counsel to the President (2009-2011); Visiting Fellow, The Brookings Institution
Bruce Freed, Founder and President, Center for Political Accountability
Steve Israel, Member, U.S. House of Representatives (D-NY)
Roger Katz, Chair, Government Oversight Committee, Maine State Senate (R)
Allen Loughry, Justice, Supreme Court of Appeals of West Virginia
Chuck Merin, Executive Vice President, Prime Policy Group; Lobbyist
Connie Morella, Ambassador to OECD (2003-2007); Member, U.S. House of Representatives (R-Md., 1987-2003)
Jeffrey Peck, Principal, Peck Madigan Jones; Lobbyist
Nick Penniman, Executive Director, Issue One
Trevor Potter, Commissioner, Federal Election Commission (1991-1995; Chairman,1994)
John Pudner, Executive Director, Take Back Our Republic
Ann Ravel, Commissioner, Federal Election Commission (Chairwoman, 2015)
Timothy Roemer, Ambassador to India (2009-2011); Member, U.S. House of Representatives (D-Ind., 1991-2003); member 9/11 Commission; Senior Strategic Advisor to Issue One
John Sarbanes, Member, U.S. House of Representatives (D-Md.)
Claudine Schneider, Member, U.S. House of Representatives (R-R.I.,1981-1991)
Peter Schweizer, President, Government Accountability Institute
Zephyr Teachout, CEO, Mayday PAC
Lucas Welch, Executive Director, The Pluribus Project
Fred Wertheimer, Founder and President, Democracy 21
Tim Wirth, Member, U.S. Senate (D-Colo.,1987-1993); Member, U.S. House of Representatives (D-Colo.,1975-1987)
Dan Wolf, Chair, Committee on Steering and Policy, Massachusetts State Senate (D)

Click here for a full agenda.

Video

• The campaign finance crisis in America and how to fix it - Part 1
• The campaign finance crisis in America and how to fix it - Part 2
• The campaign finance crisis in America and how to fix it - Part 3
• The campaign finance crisis in America and how to fix it - Part 4

Audio

• The campaign finance crisis in America and how to fix it: A solutions summit (Part 1)
• The campaign finance crisis in America and how to fix it: A solutions summit (Part 2)
• The campaign finance crisis in America and how to fix it: A solutions summit (Part 3)
• The campaign finance crisis in America and how to fix it: A solutions summit (Part 4)

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• Solution Summits Agenda120
• 20160121_campaign_finance_summit_transcript

Editor's Note: The introduction to the 2015 Brown Center Report on American Education appears below. Use the Table of Contents to navigate through the report online, or download a PDF of the full report.

TABLE OF CONTENTS

Part I: Girls, Boys, and Reading

Part II: Measuring Effects of the Common Core

Part III: Student Engagement

INTRODUCTION

The 2015 Brown Center Report (BCR) represents the 14^th edition of the series since the first issue was published in 2000.  It includes three studies.  Like all previous BCRs, the studies explore independent topics but share two characteristics: they are empirical and based on the best evidence available.  The studies in this edition are on the gender gap in reading, the impact of the Common Core State Standards -- English Language Arts on reading achievement, and student engagement.

Part one examines the gender gap in reading.  Girls outscore boys on practically every reading test given to a large population.  And they have for a long time.  A 1942 Iowa study found girls performing better than boys on tests of reading comprehension, vocabulary, and basic language skills.  Girls have outscored boys on every reading test ever given by the National Assessment of Educational Progress (NAEP)—the first long term trend test was administered in 1971—at ages nine, 13, and 17.  The gap is not confined to the U.S.  Reading tests administered as part of the Progress in International Reading Literacy Study (PIRLS) and the Program for International Student Assessment (PISA) reveal that the gender gap is a worldwide phenomenon.  In more than sixty countries participating in the two assessments, girls are better readers than boys. 

Perhaps the most surprising finding is that Finland, celebrated for its extraordinary performance on PISA for over a decade, can take pride in its high standing on the PISA reading test solely because of the performance of that nation’s young women.  With its 62 point gap, Finland has the largest gender gap of any PISA participant, with girls scoring 556 and boys scoring 494 points (the OECD average is 496, with a standard deviation of 94).   If Finland were only a nation of young men, its PISA ranking would be mediocre.

Part two is about reading achievement, too. More specifically, it’s about reading and the English Language Arts standards of the Common Core (CCSS-ELA).  It’s also about an important decision that policy analysts must make when evaluating public policies—the determination of when a policy begins. How can CCSS be properly evaluated? 

Two different indexes of CCSS-ELA implementation are presented, one based on 2011 data and the other on data collected in 2013.  In both years, state education officials were surveyed about their Common Core implementation efforts.  Because forty-six states originally signed on to the CCSS-ELA—and with at least forty still on track for full implementation by 2016—little variability exists among the states in terms of standards policy.  Of course, the four states that never adopted CCSS-ELA can serve as a small control group.  But variation is also found in how the states are implementing CCSS.  Some states are pursuing an array of activities and aiming for full implementation earlier rather than later.  Others have a narrow, targeted implementation strategy and are proceeding more slowly. 

The analysis investigates whether CCSS-ELA implementation is related to 2009-2013 gains on the fourth grade NAEP reading test.  The analysis cannot verify causal relationships between the two variables, only correlations.  States that have aggressively implemented CCSS-ELA (referred to as “strong” implementers in the study) evidence a one to one and one-half point larger gain on the NAEP scale compared to non-adopters of the standards.  This association is similar in magnitude to an advantage found in a study of eighth grade math achievement in last year’s BCR.  Although positive, these effects are quite small.  When the 2015 NAEP results are released this winter, it will be important for the fate of the Common Core project to see if strong implementers of the CCSS-ELA can maintain their momentum.

Part three is on student engagement.  PISA tests fifteen-year-olds on three subjects—reading, math, and science—every three years.  It also collects a wealth of background information from students, including their attitudes toward school and learning.  When the 2012 PISA results were released, PISA analysts published an accompanying volume, Ready to Learn: Students’ Engagement, Drive, and Self-Beliefs, exploring topics related to student engagement.

Part three provides secondary analysis of several dimensions of engagement found in the PISA report.  Intrinsic motivation, the internal rewards that encourage students to learn, is an important component of student engagement.  National scores on PISA’s index of intrinsic motivation to learn mathematics are compared to national PISA math scores.  Surprisingly, the relationship is negative.  Countries with highly motivated kids tend to score lower on the math test; conversely, higher-scoring nations tend to have less-motivated kids. 

The same is true for responses to the statements, “I do mathematics because I enjoy it,” and “I look forward to my mathematics lessons.”  Countries with students who say that they enjoy math or look forward to their math lessons tend to score lower on the PISA math test compared to countries where students respond negatively to the statements.  These counterintuitive finding may be influenced by how terms such as “enjoy” and “looking forward” are interpreted in different cultures.  Within-country analyses address that problem.  The correlation coefficients for within-country, student-level associations of achievement and other components of engagement run in the anticipated direction—they are positive.  But they are also modest in size, with correlation coefficients of 0.20 or less. 

Policymakers are interested in questions requiring analysis of aggregated data—at the national level, that means between-country data.  When countries increase their students’ intrinsic motivation to learn math, is there a concomitant increase in PISA math scores?  Data from 2003 to 2012 are examined.  Seventeen countries managed to increase student motivation, but their PISA math scores fell an average of 3.7 scale score points.  Fourteen countries showed no change on the index of intrinsic motivation—and their PISA scores also evidenced little change.  Eight countries witnessed a decline in intrinsic motivation.  Inexplicably, their PISA math scores increased by an average of 10.3 scale score points.  Motivation down, achievement up.

Correlation is not causation.  Moreover, the absence of a positive correlation—or in this case, the presence of a negative correlation—is not refutation of a possible positive relationship.  The lesson here is not that policymakers should adopt the most effective way of stamping out student motivation.  The lesson is that the level of analysis matters when analyzing achievement data.  Policy reports must be read warily—especially those freely offering policy recommendations.  Beware of analyses that exclusively rely on within- or between-country test data without making any attempt to reconcile discrepancies at other levels of analysis.  Those analysts could be cherry-picking the data.  Also, consumers of education research should grant more credence to approaches modeling change over time (as in difference in difference models) than to cross-sectional analyses that only explore statistical relationships at a single point in time. 

Downloads

• Download the report

Tom Loveless, a nonresident senior fellow in Governance Studies, explains his latest research on measuring achievement of American students.

“The bottom line here: the implementation of the common core has appeared to have very little impact on student achievement,” Loveless says. In this episode, he discusses whether the common core is failing our students, whether AP achievement is indicative of student success, and the role of principals as instructional leaders.

Also in this episode: Get to know Constanze Stelzenmüller, the Robert Bosch Senior Fellow in the Center on the United States and Europe, during our "Coffee Break” segment. Also stay tuned to hear the final episode in our centenary series with current and past Brookings scholars.

Show Notes:

The Brown Center Report on American Education

Brookings Centenary Timeline

Subscribe to the Brookings Cafeteria on iTunes, listen in all the usual places, and send feedback email to BCP@Brookings.edu.

The 2016 Brown Center Report (BCR), which was published last week, presented a study of Common Core State Standards (CCSS).   In this post, I’d like to elaborate on a topic touched upon but deserving further attention: what to expect in Common Core’s immediate political future. I discuss four key challenges that CCSS will face between now and the end of the year.

Let’s set the stage for the discussion.  The BCR study produced two major findings.  First, several changes that CCSS promotes in curriculum and instruction appear to be taking place at the school level.  Second, states that adopted CCSS and have been implementing the standards have registered about the same gains and losses on NAEP as states that either adopted and rescinded CCSS or never adopted CCSS in the first place.  These are merely associations and cannot be interpreted as saying anything about CCSS’s causal impact.  Politically, that doesn’t really matter. The big story is that NAEP scores have been flat for six years, an unprecedented stagnation in national achievement that states have experienced regardless of their stance on CCSS.  Yes, it’s unfair, but CCSS is paying a political price for those disappointing NAEP scores.  No clear NAEP differences have emerged between CCSS adopters and non-adopters to reverse that political dynamic.

"Yes, it’s unfair, but CCSS is paying a political price for those disappointing NAEP scores. No clear NAEP differences have emerged between CCSS adopters and non-adopters to reverse that political dynamic."

TIMSS and PISA scores in November-December

NAEP has two separate test programs.  The scores released in 2015 were for the main NAEP, which began in 1990.  The long term trend (LTT) NAEP, a different test that was first given in 1969, has not been administered since 2012.  It was scheduled to be given in 2016, but was cancelled due to budgetary constraints.  It was next scheduled for 2020, but last fall officials cancelled that round of testing as well, meaning that the LTT NAEP won’t be given again until 2024.  

With the LTT NAEP on hold, only two international assessments will soon offer estimates of U.S. achievement that, like the two NAEP tests, are based on scientific sampling:  PISA and TIMSS.  Both tests were administered in 2015, and the new scores will be released around the Thanksgiving-Christmas period of 2016.  If PISA and TIMSS confirm the stagnant trend in U.S. achievement, expect CCSS to take another political hit.  America’s performance on international tests engenders a lot of hand wringing anyway, so the reaction to disappointing PISA or TIMSS scores may be even more pronounced than what the disappointing NAEP scores generated.

Is teacher support still declining?

Watch Education Next’s survey on Common Core (usually released in August/September) and pay close attention to teacher support for CCSS.  The trend line has been heading steadily south. In 2013, 76 percent of teachers said they supported CCSS and only 12 percent were opposed.  In 2014, teacher support fell to 43 percent and opposition grew to 37 percent.  In 2015, opponents outnumbered supporters for the first time, 50 percent to 37 percent.  Further erosion of teacher support will indicate that Common Core’s implementation is in trouble at the ground level.  Don’t forget: teachers are the final implementers of standards.

An effort by Common Core supporters to change NAEP

The 2015 NAEP math scores were disappointing.  Watch for an attempt by Common Core supporters to change the NAEP math tests. Michael Cohen, President of Achieve, a prominent pro-CCSS organization, released a statement about the 2015 NAEP scores that included the following: "The National Assessment Governing Board, which oversees NAEP, should carefully review its frameworks and assessments in order to ensure that NAEP is in step with the leadership of the states. It appears that there is a mismatch between NAEP and all states' math standards, no matter if they are common standards or not.” 

Reviewing and potentially revising the NAEP math framework is long overdue.  The last adoption was in 2004.  The argument for changing NAEP to place greater emphasis on number and operations, revisions that would bring NAEP into closer alignment with Common Core, also has merit.  I have a longstanding position on the NAEP math framework. In 2001, I urged the National Assessment Governing Board (NAGB) to reject the draft 2004 framework because it was weak on numbers and operations—and especially weak on assessing student proficiency with whole numbers, fractions, decimals, and percentages.  

Common Core’s math standards are right in line with my 2001 complaint.  Despite my sympathy for Common Core advocates’ position, a change in NAEP should not be made because of Common Core.  In that 2001 testimony, I urged NAGB to end the marriage of NAEP with the 1989 standards of the National Council of Teachers of Mathematics, the math reform document that had guided the main NAEP since its inception.  Reform movements come and go, I argued.  NAGB’s job is to keep NAEP rigorously neutral.  The assessment’s integrity depends upon it.  NAEP was originally intended to function as a measuring stick, not as a PR device for one reform or another.  If NAEP is changed it must be done very carefully and should be rooted in the mathematics children must learn.  The political consequences of it appearing that powerful groups in Washington, DC are changing “The Nation’s Report Card” in order for Common Core to look better will hurt both Common Core and NAEP.

Will Opt Out grow?

Watch the Opt Out movement.  In 2015, several organized groups of parents refused to allow their children to take Common Core tests.  In New York state alone, about 60,000 opted out in 2014, skyrocketing to 200,000 in 2015.  Common Core testing for 2016 begins now and goes through May.  It will be important to see whether Opt Out can expand to other states, grow in numbers, and branch out beyond middle- and upper-income neighborhoods.

Conclusion

Common Core is now several years into implementation.  Supporters have had a difficult time persuading skeptics that any positive results have occurred. The best evidence has been mixed on that question.  CCSS advocates say it is too early to tell, and we’ll just have to wait to see the benefits.  That defense won’t work much longer.  Time is running out.  The political challenges that Common Core faces the remainder of this year may determine whether it survives.

On May 16, I got into a Twitter argument with Campbell Brown of The 74, an education website.  She released a video on Slate giving advice to the next president.  The video begins: “Without question, to me, the issue is education. Two out of three eighth graders in this country cannot read or do math at grade level.”  I study student achievement and was curious.  I know of no valid evidence to make the claim that two out of three eighth graders are below grade level in reading and math.  No evidence was cited in the video.  I asked Brown for the evidentiary basis of the assertion.  She cited the National Assessment of Educational Progress (NAEP).

NAEP does not report the percentage of students performing at grade level.  NAEP reports the percentage of students reaching a “proficient” level of performance.  Here’s the problem. That’s not grade level. 

In this post, I hope to convince readers of two things:

1.  Proficient on NAEP does not mean grade level performance.  It’s significantly above that.
2.  Using NAEP’s proficient level as a basis for education policy is a bad idea.

Before going any further, let’s look at some history.

NAEP history 

NAEP was launched nearly five decades ago.  The first NAEP test was given in science in 1969, followed by a reading test in 1971 and math in 1973.  For the first time, Americans were able to track the academic progress of the nation’s students.  That set of assessments, which periodically tests students 9, 13, and 17 years old and was last given in 2012, is now known as the Long Term Trend (LTT) NAEP. 

It was joined by another set of NAEP tests in the 1990s.  The Main NAEP assesses students by grade level (fourth, eighth, and twelfth) and, unlike the LTT, produces not only national but also state scores.  The two tests, LTT and main, continue on parallel tracks today, and they are often confounded by casual NAEP observers.  The main NAEP, which was last administered in 2015, is the test relevant to this post and will be the only one discussed hereafter.  The NAEP governing board was concerned that the conventional metric for reporting results (scale scores) was meaningless to the public, so achievement standards (also known as performance standards) were introduced.  The percentage of students scoring at advanced, proficient, basic, and below basic levels are reported each time the main NAEP is given.

Does NAEP proficient mean grade level? 

The National Center for Education Statistics (NCES) states emphatically, “Proficient is not synonymous with grade level performance.” The National Assessment Governing Board has a brochure with information on NAEP, including a section devoted to myths and facts.  There, you will find this:

Myth: The NAEP Proficient level is like being on grade level.

Fact: Proficient on NAEP means competency over challenging subject matter.  This is not the same thing as being “on grade level,” which refers to performance on local curriculum and standards. NAEP is a general assessment of knowledge and skills in a particular subject.

Equating NAEP proficiency with grade level is bogus.  Indeed, the validity of the achievement levels themselves is questionable.  They immediately came under fire in reviews by the U.S. Government Accountability Office, the National Academy of Sciences, and the National Academy of Education.[1]  The National Academy of Sciences report was particularly scathing, labeling NAEP’s achievement levels as “fundamentally flawed.”

Despite warnings of NAEP authorities and critical reviews from scholars, some commentators, typically from advocacy groups, continue to confound NAEP proficient with grade level.  Organizations that support school reform, such as Achieve Inc. and Students First, prominently misuse the term on their websites.  Achieve presses states to adopt cut points aligned with NAEP proficient as part of new Common Core-based accountability systems.  Achieve argues that this will inform parents whether children “can do grade level work.” No, it will not.  That claim is misleading.

How unrealistic is NAEP proficient? 

Shortly after NCLB was signed into law, Robert Linn, one of the most prominent psychometricians of the past several decades, called ”the target of 100% proficient or above according to the NAEP standards more like wishful thinking than a realistic possibility.”  History is on the side of that argument.  When the first main NAEP in mathematics was given in 1990, only 13 % of eighth graders scored proficient and 2 % scored advanced.  Imagine using “proficient” as synonymous with grade level—85 % scored below grade level! 

The 1990 national average in eighth grade scale scores was 263 (see Table 1).  In 2015, the average was 282, a gain of 19 scale score points.

Table 1.  Main NAEP Eighth Grade Math Score, by achievement levels, 1990-2015

That’s an impressive gain.  Analysts who study NAEP often use 10 points on the NAEP scale as a back of the envelope estimate of one year’s worth of learning.  Eighth graders have gained almost two years.  The percentage of students scoring below basic has dropped from 48%  in 1990 to 29% in 2015.  The percentage of students scoring proficient or above has more than doubled, from 15% to 33%.  That’s not bad news; it’s good news.

But the cut point for NAEP proficient is 299.  By that standard, two-thirds of eighth graders are still falling short.  Even students in private schools, despite hailing from more socioeconomically advantaged homes and in some cases being selectively admitted by schools, fail miserably at attaining NAEP proficiency.  More than half (53 percent) are below proficient. 

Today’s eighth graders have made it about half-way to NAEP proficient in 25 years, but they still need to gain almost two more years of math learning (17 points) to reach that level.  And, don’t forget, that’s just the national average, so even when that lofty goal is achieved, half of the nation’s students will still fall short of proficient.  Advocates of the NAEP proficient standard want it to be for all students.  That is ridiculous.  Another way to think about it: proficient for today’s eighth graders reflects approximately what the average twelfth grader knew in mathematics in 1990.   Someday the average eighth grader may be able to do that level of mathematics.  But it won’t be soon, and it won’t be every student.

In the 2007 Brown Center Report on American Education, I questioned whether NAEP proficient is a reasonable achievement standard.[2]  That year, a study by Gary Phillips of American Institutes for Research was published that projected the 2007 TIMSS scores on the NAEP scale.  Phillips posed the question: based on TIMSS, how many students in other countries would score proficient or better on NAEP?  The study’s methodology only produces approximations, but they are eye-popping.

Here are just a few countries:

Table 2.  Projected Percent NAEP Proficient, Eighth Grade Math

Singapore was the top scoring nation on TIMSS that year, but even there, more than a quarter of students fail to reach NAEP proficient.  Japan is not usually considered a slouch on international math assessments, but 43% of its eighth graders fall short.  The U.S. looks weak, with only 26% of students proficient.  But England, Israel, and Italy are even weaker.  Norway, a wealthy nation with per capita GDP almost twice that of the U.S., can only get 9 out of 100 eighth graders to NAEP proficient.

Finland isn’t shown in the table because it didn’t participate in the 2007 TIMSS.  But it did in 2011, with Finland and the U.S. scoring about the same in eighth grade math.  Had Finland’s eighth graders taken NAEP in 2011, it’s a good bet that the proportion scoring below NAEP proficient would have been similar to that in the U.S.  And yet articles such as “Why Finland Has the Best Schools,” appear regularly in the U.S. press.[3]

Why it matters

The National Center for Education Statistics warns that federal law requires that NAEP achievement levels be used on a trial basis until the Commissioner of Education Statistics determines that the achievement levels are “reasonable, valid, and informative to the public.”  As the NCES website states, “So far, no Commissioner has made such a determination, and the achievement levels remain in a trial status.  The achievement levels should continue to be interpreted and used with caution.”

Confounding NAEP proficient with grade-level is uninformed.  Designating NAEP proficient as the achievement benchmark for accountability systems is certainly not cautious use.  If high school students are required to meet NAEP proficient to graduate from high school, large numbers will fail. If middle and elementary school students are forced to repeat grades because they fall short of a standard anchored to NAEP proficient, vast numbers will repeat grades.    

On NAEP, students are asked the highest level math course they’ve taken.  On the 2015 twelfth grade NAEP, 19% of students said they either were taking or had taken calculus.   These are the nation’s best and the brightest, the crème-de la crème of math students.  Only one in five students work their way that high up the hierarchy of American math courses.  If you are over 45 years old and reading this, the proportion who took calculus in high school is less than one out of ten.  In the graduating class of 1990, for instance, only 7% of students had taken calculus.[4] 

Unsurprisingly, calculus students are also typically taught by the nation’s most knowledgeable math teachers.  The nation’s elite math students paired with the nation’s elite math teachers: if any group can prove NAEP proficient a reasonable goal and succeed in getting all students over the NAEP proficiency bar, this is the group. 

But they don’t.  A whopping 30% score below proficient on NAEP.  For black and Hispanic calculus students, the figures are staggering.  Two-thirds of black calculus students score below NAEP proficient.  For Hispanics, the figure is 52%.  The nation’s pre-calculus students also fair poorly (69% below proficient). Then the success rate falls off a cliff.  In the class of 2015, more than nine out of ten students whose highest math course was Trigonometry or Algebra II fail to meet the NAEP proficient standard.

Table 3.  2015 NAEP Twelfth Grade Math, Proficient by Highest Math Course Taken

Source: NAEP Data Explorer

These data defy reason; they also refute common sense.  For years, educators have urged students to take the toughest courses they can possibly take.  Taken at face value, the data in Table 3 rip the heart out of that advice.  These are the toughest courses, and yet huge numbers of the nation’s star students, by any standard aligned with NAEP proficient, would be told that they have failed.  Some parents, misled by the confounding of proficient with grade level, might even mistakenly believe that their kids don’t know grade level math.

Conclusion 

NAEP proficient is not synonymous with grade level.  NAEP officials urge that proficient not be interpreted as reflecting grade level work.  It is a standard set much higher than that.  Scholarly panels have reviewed the NAEP achievement standards and found them flawed.  The highest scoring nations of the world would appear to be mediocre or poor performers if judged by the NAEP proficient standard.  Even large numbers of U.S. calculus students fall short.

As states consider building benchmarks for student performance into accountability systems, they should not use NAEP proficient—or any standard aligned with NAEP proficient—as a benchmark.  It is an unreasonable expectation, one that ill serves America’s students, parents, and teachers--and the effort to improve America’s schools.

Hypocrisy is not scarce in the world of politics. But the current House and Senate budget resolutions set new lows. Each proposes to cut about $5 trillion from government spending over the next decade in pursuit of a balanced budget. Whatever one may think of putting the goal of reducing spending when the ratio of the debt-to-GDP is projected to be stable above investing in the nation’s future, you would think that deficit-reduction hawks wouldn’t cut spending that has been proven to lower the deficit.

Yes, there are expenditures that actually lower the deficit, typically by many dollars for each dollar spent. In this category are outlays on ‘program integrity’ to find and punish fraud, tax evasion, and plain old bureaucratic mistakes. You might suppose that those outlays would be spared. Guess again. Consider the following:

Medicare. Roughly 10% of Medicare’s $600 billion budget goes for what officials delicately call ‘improper payments, according to the 2014 financial report of the Department of Health and Human Services. Some are improper merely because providers ‘up-code’ legitimate services to boost their incomes. Some payments go for services that serve no valid purpose. And some go for phantom services that were never provided. Whatever the cause, approximately $60 billion of improper payments is not ‘chump change.’

Medicare tries to root out these improper payments, but it lacks sufficient staff to do the job. What it does spend on ‘program integrity’ yields an estimated $14.40? for each dollar spent, about $10 billion a year in total. That number counts only directly measurable savings, such as recoveries and claim denials. A full reckoning of savings would add in the hard-to-measure ‘policeman on the beat’ effect that discourages violations by would-be cheats.

Fat targets remain. A recent report from the Institute of Medicine presented findings that veritably scream ‘fraud.’ Per person spending on durable medical equipment and home health care is ten times higher in Miami-Dade County, Florida than the national average. Such equipment and home health accounts for nearly three-quarters of the geographical variation in per person Medicare spending. Yet, only 4% of current recoveries of improper payments come from audits of these two items and little from the highest spending locations.

Why doesn’t Medicare spend more and go after the remaining overpayments, you may wonder? The simple answer is that Congress gives Medicare too little money for administration. Direct overhead expenses of Medicare amount to only about 1.5% of program outlays—6% if one includes the internal administrative costs of private health plans that serve Medicare enrollees. Medicare doesn’t need to spend as much on administration as the average of 19% spent by private insurers, because for example, Medicare need not pay dividends to private shareholders or advertise.

But spending more on Medicare administration would both pay for itself—$2 for each added dollar spent, according to the conservative estimate in the President’s most recent budget—and improve the quality of care. With more staff, Medicare could stop more improper payments and reduce the use of approved therapies in unapproved ways that do no good and may cause harm.

Taxes. Compare two numbers: $540 billion and $468 billion. The first number is the amount of taxes owed but not paid. The second number is the projected federal budget deficit for 2015, according to the Congressional Budget Office.

Collecting all taxes legally owed but not paid is an impossibility. It just isn’t worth going after every violation. But current enforcement falls far short of practical limits. Expenditures on enforcement directly yields $4 to $6 for each dollar spent on enforcement. Indirect savings are many times larger—the cop-on-the-beat effect again. So, in an era of ostentatious concern about budget deficits, you would expect fiscal fretting in Congress to lead to increased efforts to collect what the law says people owe in taxes.

Wrong again. Between 2010 and 2014, the IRS budget was cut in real terms by 20%. At the same time, the agency had to shoulder new tasks under health reform, as well as process an avalanche of applications for tax exemptions unleashed by the 2010 Supreme Court decision in the Citizens United case. With less money to spend and more to do, enforcement staff dropped by 15% and inflation adjusted collections dropped 13%.

One should acknowledge that enforcement will not do away with most avoidance and evasion. Needlessly complex tax laws are the root cause of most tax underpayment. Tax reform would do even more than improved administration to increase the ratio of taxes paid to taxes due. But until that glorious day when Congress finds the wit and will to make the tax system simpler and fairer, it would behoove a nation trying to make ends meet to spend $2 billion to $3 billion more each year to directly collect $10 billion to 15 billion a year more of legally owed taxes and, almost certainly, raise far more than that by frightening borderline scoff-laws.

Disability Insurance. Thirteen million people with disabling conditions who are judged incapable of engaging in substantial gainful activity received $161 billion in disability insurance in 2013. If the disabling conditions improve enough so that beneficiaries can return to work, benefits are supposed to be stopped. Such improvement is rare. But when administrators believe that there is some chance, the law requires them to check. They may ask beneficiaries to fill out a questionnaire or, in some cases, undergo a new medical exam at government expense. Each dollar spent in these ways generated an estimated $16 in savings in 2013.

Still, the Social Security Administration is so understaffed that SSA has a backlog of 1.3 million disability reviews. Current estimates indicate that spending a little over $1 billion a year more on such reviews over the next decade would save $43 billion. Rather than giving Social Security the staff and spending authority to work down this backlog and realize those savings, Congress has been cutting the agency’s administrative budget and sequestration threatens further cuts.

Claiming that better administration will balance the budget would be wrong. But it would help. And it would stop some people from shirking their legal responsibilities and lighten the burdens of those who shoulder theirs. The failure of Congress to provide enough staff to run programs costing hundreds of billions of dollars a year as efficiently and honestly as possible is about as good a definition of criminal negligence as one can find.

The addition of Medicare in 1965 completed a suite of federal programs designed to protect the wealth and health of people reaching older ages in the United States, starting with the Committee on Economic Security of 1934—known today as Social Security. While few would deny Medicare’s important role in improving older and disabled Americans’ financial security and health, many worry about sustaining and strengthening Medicare to finance high-quality, affordable health care for coming generations.

In 1965, average life expectancy for a 65-year-old man and woman was another 13 years and 16 years, respectively. Now, life expectancy for 65-year-olds is 18 years for men and 20 years for women—effectively a four- to five-year increase.

In 2011, the first of 75-million-plus baby boomers became eligible for Medicare. And by 2029, when all of the baby boomers will be 65 or older, the U.S. Census Bureau predicts 20 percent of the U.S. population will be older than 65. Just by virtue of the sheer size of the aging population, Medicare spending growth will accelerate sharply in the coming years.

Event Information

In its 50th year, the Medicare program currently provides health insurance coverage for more than 49 million Americans and accounts for $600 billion in federal spending. With those numbers expected to rise as the baby boomer generation ages, many policy experts consider this impending expansion a major threat to the nation’s economic future and question how it might affect the quality and value of health care for Medicare beneficiaries.

On June 5, the Center for Health Policy at Brookings and the USC Leonard D. Schaeffer Center for Health Policy and Economics hosted a half-day forum on the future of Medicare. Instead of reflecting on historical accomplishments, the event looked ahead to 2030—a time when the youngest Baby Boomers will be Medicare-eligible—and explore the changing demographics, health care needs, medical technology costs, and financial resources available to beneficiaries. The panels focused on modernizing Medicare's infrastructure, benefit design, marketplace competition, and payment mechanisms. The event also included the release of five policy papers from featured panelists.

Please note that presentation slides from USC's Dana Goldman will not be available for download. For more information on findings from his presentation download the working paper available on this page or watch the event video.

Video

• Challenges and opportunities facing Medicare in 2030
• Eligibility, benefit design, and financial support
• Could improving choice and competition in Medicare Advantage be the future of Medicare?
• Improving provider payment in Medicare

Audio

• Strengthening Medicare for 2030

Transcript

• Uncorrected Transcript (.pdf)

Event Materials

• Burtless Slides
• 20150605_medicare_2030_transcript

Medicare Hospital Insurance and Social Security would not add to deficits because they can’t spend money they don’t have.

The dog days of August have given way to something much worse. Congress returned to session this week, and the rest of the year promises to be nightmarish. The House and Senate passed budget resolutions earlier this year calling for nearly $5 trillion in spending cuts by 2025. More than two-thirds of those cuts would come from programs that help people with low-and moderate-incomes. Health care spending would be halved. If such cuts are enacted, the president will likely veto them. At best, another partisan budget war will ensue after which the veto is sustained. At worst, the cuts become law.

The putative justification for these cuts is that the nation faces insupportable increases in public debt because of expanding budget deficits. Even if the projections were valid, it would be prudent to enact some tax increases in order to preserve needed public spending. But the projections of explosively growing debt are not valid. They are fantasy.

Wait! you say. The Congressional Budget Office has been telling us for years about the prospect of rising deficit and exploding debt. They repeated those warnings just two months ago. Private organizations of both the left and right agree with the CBO’s projections, in general if not in detail. How can any sane person deny that the nation faces a serious long-term budget deficit problem?

The answer is simple: The CBO and private organizations use a convention in preparing their projections that is at odds with established policy and law. If, instead, projections are based on actual current law, as they claim to be, the specter of an increasing debt burden vanishes. What is that convention? Why is it wrong? Why did CBO adopt it, and why have others kept it?

CBO’s budget projections cover the next 75 years. Its baseline projections claim to be based on current law and policy. (CBO also presents an ‘alternative scenario’ based on assumed changes in law and policy). Within that period, Social Security (OASDI) and Medicare Hospital Insurance (HI) expenditures are certain to exceed revenues earmarked to pay for them. Both are financed through trust funds. Both funds have sizeable reserves — government securities — that can be used to cover short falls for a while. But when those reserves are exhausted, expenditures cannot exceed current revenues. Trust fund financing means that neither Social Security nor Medicare Hospital Insurance can run deficits. Nor can they add to the public debt.

Nonetheless, CBO and other organizations assume that Social Security and Medicare Hospital Insurance can and will spend money they don’t have and that current law bars them from spending.

One of the reasons why trust fund financing was used, first for Social Security and then for Medicare Hospital Insurance, was to create a framework that disciplined Congress earmarked to earmark sufficient revenues to pay for benefits it might award. Successive presidents and Congresses, both Republican and Democratic, have repeatedly acted to prevent either program’s cumulative spending from exceeding cumulative revenues. In 1983, for example, faced with an impending trust fund shortfall, Congress cut benefits and raised taxes enough to turn prospective cash flow trust fund deficits into cash flow surpluses. And President Reagan signed the bill. In so doing, they have reaffirmed the discipline imposed by trust fund financing.

Trust fund accounting explains why people now are worrying about the adequacy of funding for Social Security and Medicare. They recognize that the trust funds will be depleted in a couple of decades. They understand that between now and then Congress must either raise earmarked taxes or fashion benefit cuts. If it doesn’t raise taxes, benefits will be cut across the board. Either way, the deficits that CBO and other organizations have built into their budget projections will not materialize.

The implications for projected debt of CBO’s inclusion in its projections of deficits that current law and established policy do not allow are enormous, as the graph below shows.

If one excludes deficits in Social Security and Medicare Hospital Insurance that cannot occur under current law and established policy, the ratio of national debt to gross domestic product will fall, not rise, as CBO budget projections indicate. In other words, the claim that drastic cuts in government spending are necessary to avoid calamitous budget deficits is bogus.

It might seem puzzling that CBO, an agency known for is professionalism and scrupulous avoidance of political bias, would adopt a convention so at odds with law and policy. The answer is straightforward—Congress makes them do it. Section 257 of the Balanced Budget and Emergency Deficit Control Act of 1985 requires CBO to assume that the trust funds can spend money although legislation governing trust fund operations bars such expenditures. CBO is obeying the law.

No similar explanation exonerates the statement of the Committee for a Responsible Federal Budget, which on August 25, 2015 cited, with approval, the conclusion that ‘debt continues to grow unsustainably,’ or that of the Bipartisan Policy Center, which wrote on the same day that ‘America’s debt continues to grow on an unsustainable path.’ Both statements are wrong.

To be sure, the dire budget future anticipated in the CBO projections could materialize. Large deficits could result from an economic calamity or war. Congress could abandon the principle that Social Security and Medicare Hospital Insurance should be financed within trust funds. It could enact other fiscally rash policies. But such deficits do not flow from current law or reflect the trust fund discipline endorsed by both parties over the last 80 years. And it is current law and policy that are supposed to underlie budget projections. Slashing spending because a thirty-year old law requires CBO to assume that Congress will do something it has shown no sign of doing—overturn decades of bipartisan prudence requiring that the major social insurance programs spend only money specifically earmarked for them, and not a penny more—would impose enormous hardship on vulnerable populations in the name of a fiscal fantasy.

Editor's Note: This post originally appeared in Fortune Magazine.

As government funding for disability insurance is expected to run out next year, Congress should re-evaluate the costs of the program.

Nine million people in America today are receiving Social Security Disability Insurance, double the number in 1995 and six times the number in 1970. With statistics like that, it’s hardly surprising to see some in Congress worry that more will enroll in the program and costs would continue to rise, especially since government funding for disability insurance is expected to run out by the end of next year. If Congress does nothing, benefits would fall by 19% immediately following next year’s presidential election. So, Congress will likely do something. But what exactly should it do?

Funding for disability insurance has nearly run out of money before. Each time, Congress has simply increased the share of the Social Security payroll tax that goes for disability insurance. This time, however, many members of Congress oppose such a shift unless it is linked to changes that curb eligibility and promote return to work. They fear that rolls will keep growing and costs would keep rising, but findings from a report by a government panel conclude that disability insurance rolls have stopped rising and will likely shrink. The report, authored by a panel of the Social Security Advisory Board, is important in that many of the factors that caused disability insurance to rise, particularly during the Great Recession, have ended.

• Baby-boomers, who added to the rolls as they reached the disability-prone middle age years, are aging out of disability benefits and into retirement benefits. 


• The decades-long flood of women increased the pool of people with the work histories needed to be eligible for disability insurance. But women’s labor force participation has fallen a bit from pre-Great Recession peaks, and is not expected again to rise materially. 


• The Great Recession, which led many who lost jobs and couldn’t find work to apply for disability insurance, is over and applications are down. A recession as large as that of 2008 is improbable any time soon. 


• Approval rates by administrative law judges, who for many years were suspected of being too ready to approve applications, have been falling. Whatever the cause, this stringency augurs a fall in the disability insurance rolls.

Two recently posted papers by Brookings colleagues purport to show that “even a large increase in the top marginal rate would barely reduce inequality.”[1]  This conclusion, based on one commonly used measure of inequality, is an incomplete and misleading answer to the question posed: would a stand-alone increase in the top income tax bracket materially reduce inequality?  More importantly, it is the wrong question to pose, as a stand-alone increase in the top bracket rate would be bad tax policy that would exacerbate tax avoidance incentives.  Sensible tax policy would package that change with at least one other tax modification, and such a package would have an even more striking effect on income inequality.  In brief:

• A stand-alone increase in the top tax bracket would be bad tax policy, but it would meaningfully increase the degree to which the tax system reduces economic inequality.  It would have this effect even though it would fall on just ½ of 1 percent of all taxpayers and barely half of their income.
• Tax policy significantly reduces inequality.  But transfer payments and other spending reduce it far more.  In combination, taxes and public spending materially offset the inequality generated by market income.
• The revenue from a well-crafted increase in taxes on upper-income Americans, dedicated to a prudent expansions of public spending, would go far to counter the powerful forces that have made income inequality more extreme in the United States than in any other major developed economy.

---

Downloads

• Download the paper

Social Security faces a financial shortfall. If Congress does nothing about it, current projections indicate that benefits will be cut automatically by 21 percent in 2034. Congress could close the gap by raising revenues, lowering benefits, or doing some of both. If benefits seem generous, Congress is likely to lean toward benefit cuts more than revenue increases. If they seem stingy, then the reverse.

Given the split between the two parties on whether to cut benefits or to raise them, evidence on the adequacy of benefits is central to this key policy debate. Those perceptions will help determine whether Social Security continues to provide basic retirement income for workers with comparatively low earnings histories and a foundation of retirement income for most others or it will become just a minimal safety-net backstop against extreme destitution?

Down-in-the-weeds disagreements among analysts often seem too arcane for anyone other than specialists. But sometimes they are too important to ignore. A current debate about the adequacy of Social Security benefits is an example.

The not-so-simple question is this: are Social Security benefits ‘generous’ or ‘stingy’? To answer this question, people long looked to the Office of the Social Security Actuary. For many years that office published estimates of something called the ‘replacement rate’—that is, how high are benefits paid to retirees and the disabled relative what they earned during their working years. A 2014 retiree with median earnings had average lifetime earnings of about $46,000. That worker qualified for a benefit at age 66 of about $19,000, a replacement rate of about 41%. Replacement rates vary with earnings. Dollar benefits rise with earnings, but they rise less than proportionately. As a result, replacement rates of low earners are higher than replacement rates of high earners.

As you might suppose, there are many ways in which to compute such ‘replacement rates. Because of analytical disputes on which method is best, the Social Security trustees in 2014 decided to stop including replacement rate estimates in their annual reports.

In December 2015, the Congressional Budget Office (CBO) offered what it considered a better measure of the generosity of Social Security. It estimated that replacement rates for middle income recipients were about 60%–dramatically higher than the 41% that the Social Security Trustees had estimated.

The gap between the estimates of CBO and those of Social Security is even larger than it seems. To see why, one needs to recognize that to sustain living standards retirees on average need only about 75% to 80% as much income as they did when working. Retirees need less income because they are spared some work-related expenses, such as transportation to and from work. Those are only average of course; some need more, some less.

If one believed the SSA actuaries, Social Security provides median earners barely more than half of what they need to be as well off as they were when working. Benefit cuts from that modest level would threaten the well-being for the majority of retirees who are entirely or mostly dependent on Social Security benefits—and especially for those with large medical expenses uncovered by Medicare.

On the other hand, if one accepted CBO’s estimates, Social Security provids more than three-quarters of the retirement income target. Against that baseline, benefit cuts would still sting, but they would pose less of a threat, and not much of a threat at all for most retirees who have some income from private pensions or personal savings.

When the CBO estimates came out, conservative commentators welcomed the findings and cited CBO’s well-established and well-earned reputation for objectivity. They correctly noted that many retirees have additional income from private pensions, 401ks, or other personal savings, and asserted that there was no general retirement income shortage. By inference, cutting benefits a bit to help close the long-term funding gap would be no big deal. Social Security advocates were put on the defensive, hard-pressed to challenge the estimates of the widely-respected Congressional Budget Office.

But earlier this year, CBO acknowledged that it had made mistakes in its Decameter estimates and revised them. The new CBO estimate put the replacement rate for middle-level earners at around 42%, almost the same as the estimate of the Social Security actuaries, not the much higher level that had sent ripples through the policy community. One conservative analyst, Andrew Biggs, who had trumpeted the initial CBO finding in The Wall Street Journal, promptly and honorably retracted his article.

Two aspects of this green-eyeshade kerfuffle stand out. The first is that policy debates often depend on obscure technical analyses that are, in turn, remarkably sensitive to ‘black-box’ methods to which few or no outsiders have ready access. The second is that CBO burnished its reputation for honesty by owning up to its own mistakes — in this case, a whopping overestimate of a key number. Such candor is all too rare; it merits notice and praise.

But there is a broader lesson as well. Technical issues of comparable complexity surround numerous current political disputes. Is Bernie Sanders’ single-payer plan affordable? Will Marco Rubio’s tax plan cause deficits to balloon? To vote rationally, people must struggle to see through the rhetorical chaff that surrounds candidates’ favorite claims. There is, alas, no substitute for paying close attention to the data, even if they are ‘down in the weeds.’

Editor's note: This piece originally appeared in Fortune.