Earlier this month, the College Board released SAT scores for the high school graduating class of 2015. Both math and reading scores declined from 2014, continuing a steady downward trend that has been in place for the past decade. Pundits of contrasting political stripes seized on the scores to bolster their political agendas. Michael Petrilli of the Fordham Foundation argued that falling SAT scores show that high schools need more reform, presumably those his organization supports, in particular, charter schools and accountability.* For Carol Burris of the Network for Public Education, the declining scores were evidence of the failure of polices her organization opposes, namely, Common Core, No Child Left Behind, and accountability.

Petrilli and Burris are both misusing SAT scores. The SAT is not designed to measure national achievement; the score losses from 2014 were miniscule; and most of the declines are probably the result of demographic changes in the SAT population. Let’s examine each of these points in greater detail.

It never was. The SAT was originally meant to measure a student’s aptitude for college independent of that student’s exposure to a particular curriculum. The test’s founders believed that gauging aptitude, rather than achievement, would serve the cause of fairness. A bright student from a high school in rural Nebraska or the mountains of West Virginia, they held, should have the same shot at attending elite universities as a student from an Eastern prep school, despite not having been exposed to the great literature and higher mathematics taught at prep schools. The SAT would measure reasoning and analytical skills, not the mastery of any particular body of knowledge. Its scores would level the playing field in terms of curricular exposure while providing a reasonable estimate of an individual’s probability of success in college.

Note that even in this capacity, the scores never suffice alone; they are only used to make admissions decisions by colleges and universities, including such luminaries as Harvard and Stanford, in combination with a lot of other information—grade point averages, curricular resumes, essays, reference letters, extra-curricular activities—all of which constitute a student’s complete application.

Today’s SAT has moved towards being a content-oriented test, but not entirely. Next year, the College Board will introduce a revised SAT to more closely reflect high school curricula. Even then, SAT scores should not be used to make judgements about U.S. high school performance, whether it’s a single high school, a state’s high schools, or all of the high schools in the country. The SAT sample is self-selected. In 2015, it only included about one-half of the nation’s high school graduates: 1.7 million out of approximately 3.3 million total. And that’s about one-ninth of approximately 16 million high school students.  Generalizing SAT scores to these larger populations violates a basic rule of social science. The College Board issues a warning when it releases SAT scores: “Since the population of test takers is self-selected, using aggregate SAT scores to compare or evaluate teachers, schools, districts, states, or other educational units is not valid, and the College Board strongly discourages such uses.”  

TIME’s coverage of the SAT release included a statement by Andrew Ho of Harvard University, who succinctly makes the point: “I think SAT and ACT are tests with important purposes, but measuring overall national educational progress is not one of them.”

SAT scores changed very little from 2014 to 2015. Reading scores dropped from 497 to 495. Math scores also fell two points, from 513 to 511. Both declines are equal to about 0.017 standard deviations (SD).^[i] To illustrate how small these changes truly are, let’s examine a metric I have used previously in discussing test scores. The average American male is 5’10” in height with a SD of about 3 inches. A 0.017 SD change in height is equal to about 1/20 of an inch (0.051). Do you really think you’d notice a difference in the height of two men standing next to each other if they only differed by 1/20^th of an inch? You wouldn’t. Similarly, the change in SAT scores from 2014 to 2015 is trivial.^[ii]

A more serious concern is the SAT trend over the past decade. Since 2005, reading scores are down 13 points, from 508 to 495, and math scores are down nine points, from 520 to 511. These are equivalent to declines of 0.12 SD for reading and 0.08 SD for math.^[iii] Representing changes that have accumulated over a decade, these losses are still quite small. In the Washington Post, Michael Petrilli asked “why is education reform hitting a brick wall in high school?” He also stated that “you see this in all kinds of evidence.”

You do not see a decline in the best evidence, the National Assessment of Educational Progress (NAEP). Contrary to the SAT, NAEP is designed to monitor national achievement. Its test scores are based on a random sampling design, meaning that the scores can be construed as representative of U.S. students. NAEP administers two different tests to high school age students, the long term trend (LTT NAEP), given to 17-year-olds, and the main NAEP, given to twelfth graders.

Table 1 compares the past ten years’ change in test scores of the SAT with changes in NAEP.^[iv] The long term trend NAEP was not administered in 2005 or 2015, so the closest years it was given are shown. The NAEP tests show high school students making small gains over the past decade. They do not confirm the losses on the SAT.

Table 1. Comparison of changes in SAT, Main NAEP (12^th grade), and LTT NAEP (17-year-olds) scores. Changes expressed as SD units of base year.

 *p<.05

Petrilli raised another concern related to NAEP scores by examining cohort trends in NAEP scores. The trend for the 17-year-old cohort of 2012, for example, can be constructed by using the scores of 13-year-olds in 2008 and 9-year-olds in 2004. By tracking NAEP changes over time in this manner, one can get a rough idea of a particular cohort’s achievement as students grow older and proceed through the school system. Examining three cohorts, Fordham’s analysis shows that the gains between ages 13 and 17 are about half as large as those registered between ages nine and 13. Kids gain more on NAEP when they are younger than when they are older.

There is nothing new here. NAEP scholars have been aware of this phenomenon for a long time. Fordham points to particular elements of education reform that it favors—charter schools, vouchers, and accountability—as the probable cause. It is true that those reforms more likely target elementary and middle schools than high schools. But the research literature on age discrepancies in NAEP gains (which is not cited in the Fordham analysis) renders doubtful the thesis that education policies are responsible for the phenomenon.^[v]

Whether high school age students try as hard as they could on NAEP has been pointed to as one explanation. A 1996 analysis of NAEP answer sheets found that 25-to-30 percent of twelfth graders displayed off-task test behaviors—doodling, leaving items blank—compared to 13 percent of eighth graders and six percent of fourth graders. A 2004 national commission on the twelfth grade NAEP recommended incentives (scholarships, certificates, letters of recognition from the President) to boost high school students’ motivation to do well on NAEP. Why would high school seniors or juniors take NAEP seriously when this low stakes test is taken in the midst of taking SAT or ACT tests for college admission, end of course exams that affect high school GPA, AP tests that can affect placement in college courses, state accountability tests that can lead to their schools being deemed a success or failure, and high school exit exams that must be passed to graduate?^[vi]

Other possible explanations for the phenomenon are: 1) differences in the scales between the ages tested on LTT NAEP (in other words, a one-point gain on the scale between ages nine and 13 may not represent the same amount of learning as a one-point gain between ages 13 and 17); 2) different rates of participation in NAEP among elementary, middle, and high schools;^[vii] and 3) social trends that affect all high school students, not just those in public schools. The third possibility can be explored by analyzing trends for students attending private schools. If Fordham had disaggregated the NAEP data by public and private schools (the scores of Catholic school students are available), it would have found that the pattern among private school students is similar—younger students gain more than older students on NAEP. That similarity casts doubt on the notion that policies governing public schools are responsible for the smaller gains among older students.^[viii]

Writing in the Washington Post, Carol Burris addresses the question of whether demographic changes have influenced the decline in SAT scores. She concludes that they have not, and in particular, she concludes that the growing proportion of students receiving exam fee waivers has probably not affected scores. She bases that conclusion on an analysis of SAT participation disaggregated by level of family income. Burris notes that the percentage of SAT takers has been stable across income groups in recent years. That criterion is not trustworthy. About 39 percent of students in 2015 declined to provide information on family income. The 61 percent that answered the family income question are probably skewed against low-income students who are on fee waivers (the assumption being that they may feel uncomfortable answering a question about family income).^[ix] Don’t forget that the SAT population as a whole is a self-selected sample. A self-selected subsample from a self-selected sample tells us even less than the original sample, which told us almost nothing.

The fee waiver share of SAT takers increased from 21 percent in 2011 to 25 percent in 2015. The simple fact that fee waivers serve low-income families, whose children tend to be lower-scoring SAT takers, is important, but not the whole story here. Students from disadvantaged families have always taken the SAT. But they paid for it themselves. If an additional increment of disadvantaged families take the SAT because they don’t have to pay for it, it is important to consider whether the new entrants to the pool of SAT test takers possess unmeasured characteristics that correlate with achievement—beyond the effect already attributed to socioeconomic status.

Robert Kelchen, an assistant professor of higher education at Seton Hall University, calculated the effect on national SAT scores of just three jurisdictions (Washington, DC, Delaware, and Idaho) adopting policies of mandatory SAT testing paid for by the state. He estimated that these policies explain about 21 percent of the nationwide decline in test scores between 2011 and 2015. He also notes that a more thorough analysis, incorporating fee waivers of other states and districts, would surely boost that figure. Fee waivers in two dozen Texas school districts, for example, are granted to all juniors and seniors in high school. And all students in those districts (including Dallas and Fort Worth) are required to take the SAT beginning in the junior year. Such universal testing policies can increase access and serve the cause of equity, but they will also, at least for a while, lead to a decline in SAT scores.

Here, I offer my own back of the envelope calculation of the relationship of demographic changes with SAT scores. The College Board reports test scores and participation rates for nine racial and ethnic groups.^[x] These data are preferable to family income because a) almost all students answer the race/ethnicity question (only four percent are non-responses versus 39 percent for family income), and b) it seems a safe assumption that students are more likely to know their race or ethnicity compared to their family’s income.

The question tackled in Table 2 is this: how much would the national SAT scores have changed from 2005 to 2015 if the scores of each racial/ethnic group stayed exactly the same as in 2005, but each group’s proportion of the total population were allowed to vary? In other words, the scores are fixed at the 2005 level for each group—no change. The SAT national scores are then recalculated using the 2015 proportions that each group represented in the national population.

Table 2. SAT Scores and Demographic Changes in the SAT Population (2005-2015)

The data suggest that two-thirds to three-quarters of the SAT score decline from 2005 to 2015 is associated with demographic changes in the test-taking population. The analysis is admittedly crude. The relationships are correlational, not causal. The race/ethnicity categories are surely serving as proxies for a bundle of other characteristics affecting SAT scores, some unobserved and others (e.g., family income, parental education, language status, class rank) that are included in the SAT questionnaire but produce data difficult to interpret.

Using an annual decline in SAT scores to indict high schools is bogus. The SAT should not be used to measure national achievement. SAT changes from 2014-2015 are tiny. The downward trend over the past decade represents a larger decline in SAT scores, but one that is still small in magnitude and correlated with changes in the SAT test-taking population.

In contrast to SAT scores, NAEP scores, which are designed to monitor national achievement, report slight gains for 17-year-olds over the past ten years. It is true that LTT NAEP gains are larger among students from ages nine to 13 than from ages 13 to 17, but research has uncovered several plausible explanations for why that occurs. The public should exercise great caution in accepting the findings of test score analyses. Test scores are often misinterpreted to promote political agendas, and much of the alarmist rhetoric provoked by small declines in scores is unjustified.

Tomorrow, President Obama unveils his last budget, and we’re sure to see plenty of proposals for spending on education and skills. In the past, the Administration has focused on investments in early childhood education, community colleges, and infrastructure and research. From a budgetary standpoint, the problem with these investments is how to capture their benefits as well as their costs.

First step: find out what works. The Obama Administration has been emphatic about the need for solid evidence in deciding what to fund. The good news is that we now have quite a lot of it, showing that investing in human capital from early education through college can make a difference. Not all programs are successful, of course, and we are still learning what works and what doesn’t. But we know enough to conclude that investing in a variety of health, education, and mobility programs can positively affect education, employment, and earnings in adulthood.

For example:

1. Young, low-income children whose families move to better neighborhoods using housing vouchers see a 31 percent increase in earnings;

2. Quality early childhood and school reform programs can raise lifetime income per child by an average of about $200,000, for at an upfront cost of about $20,000;

3. Boosting college completion rates, for instance via the Accelerated Study in Associate Programs (ASAP) in the City University of New York, leads to higher earnings.

If such estimates are correct (and we recognize there are uncertainties), policymakers are probably underinvesting in such programs because they are looking at the short-term costs but not at longer-term benefits and budget savings.

First, the CBO’s standard practice is to use a 10-year budget window, which means long-range effects are often ignored. Second, although the CBO does try to take into account behavioral responses, such as increased take-up rates of a program, or improved productivity and earnings, it often lacks the research needed to make such estimates. Third, the usual assumption is that the rate of return on public investments in human capital is less than that for private investment. This is now questionable, especially given low interest rates.

A hot topic in budget politics right now is so-called “dynamic scoring.” This means incorporating macroeconomic effects, such as an increase in the labor force or productivity gains, into cost estimates. In 2015, the House adopted a rule requiring such scoring, when practicable, for major legislation. But appropriations bills are excluded, and quantitative analyses are restricted to the existing 10-year budget window.

The interest in dynamic scoring is currently strongest among politicians pushing major tax bills, on the grounds that tax cuts could boost growth. But the principles behind dynamic scoring apply equally to improvements in productivity that could result from proposals to subsidize college education, for example—as proposed by both Senator Sanders and Secretary Clinton. Of course, it is tough to estimate the value of these potential benefits. But it is worth asking whether current budget rules lead to myopia in our assessments of what such investments might accomplish, and thus to an over-statement of their “true” cost.

The global consumer products company Unilever Plc announced on Monday a $35 million investment in Cuba’s Special Development Zone at Mariel. Late last year, Brascuba, a joint venture with a Brazilian firm, Souza Cruz, owned by the mega-conglomerate British American Tobacco (BAT), confirmed it would built a $120 million facility in the same location.

So far, these are the two biggest investments in the much-trumpeted Cuban effort to attract foreign investment, outside of traditional tourism. Yet, neither investment is really new. Unilever had been operating in Cuba since the mid-1990s, only to exit a few years ago in a contract dispute with the Cuban authorities. Brascuba will be moving its operations from an existing factory to the ZED Mariel site.

What is new is the willingness of Cuban authorities to accede to the corporate requirements of foreign investors. Finally, the Cubans appear to grasp that Cuba is a price-taker, and that it must fit into the global strategies of their international business partners. Certainly, Cuban negotiators can strike smart deals, but they cannot dictate the over-arching rules of the game.

Cuba still has a long way to go before it reaches the officially proclaimed goal of $2.5 billion in foreign investment inflows per year. Total approvals last year for ZED Mariel reached only some $200 million, and this year are officially projected to reach about $400 million. For many potential investors, the business climate remains too uncertain, and the project approval process too opaque and cumbersome. But the Brascuba and Unilever projects are definitely movements in the right direction.

But Unilever feared that granting its Cuban partner 51% would yield too much management control and could jeopardize brand quality. Unilever also balked at exporting products made in Cuba, where product costs were as much as one-third higher than in bigger Unilever plants in other Latin American countries.

The 2012 collapse of the Unilever contract renewal negotiations adversely affected investor perceptions of the business climate. If the Cuban government could not sustain a good working relationship with Unilever—a highly regarded, marquée multinational corporation with a global footprint—what international investor (at least one operating in the domestic consumer goods markets) could be confident of its ability to sustain a profitable long-term operation in Cuba?

In the design of the new joint venture, Cuba has allowed Unilever a majority 60% stake. Furthermore, in the old joint venture, Unilever executives complained that low salaries, as set by the government, contributed to low labor productivity. In ZED Mariel, worker salaries will be significantly higher: firms like Unilever will continue to pay the same wages to the government employment entity, but the entity’s tax will be significantly smaller, leaving a higher take-home pay for the workers. Hiring and firing will remain the domain of the official entity, however, not the joint venture.

Unilever is also looking forward to currency unification, widely anticipated for 2016. Previously, Unilever had enjoyed comfortable market shares in the hard-currency Cuban convertible currency (CUC) market, but had been largely excluded from the national currency markets, which state-owned firms had reserved for themselves. With currency unification, Unilever will be able to compete head-to-head with state-owned enterprises in a single national market.

Similarly, Brascuba will benefit from the new wage regime at Mariel and, as a consumer products firm, from currency unification. At its old location, Brascuba considered motivating and retaining talent to be among the firm’s key challenges; the higher wages in ZED Mariel will help to attract and retain high-quality labor.

Brascuba believes this is a good time for expansion. Better-paid workers at Mariel will be well motivated, and the expansion of the private sector is putting more money into consumer pockets. The joint venture will close its old facility in downtown Havana, in favor of the new facility at Mariel, sharply expanding production for both the domestic and international markets (primarily, Brazil).

A further incentive for investment today is the prospect of the lifting of U.S. economic sanctions, even if the precise timing is impossible to predict. Brascuba estimated that U.S. economic sanctions have raised its costs of doing business by some 20%. Inputs such as cigarette filters, manufacturing equipment and spare parts, and infrastructure such as information technology, must be sourced from more distant and often less cost-efficient sources.

Another sign of enhanced Cuban flexibility: neither investment is in a high technology sector, the loudly touted goal of ZED Mariel. A manufacturer of personal hygiene and home care product lines, Unilever will churn out toothpaste and soap, among other items. Brascuba will produce cigarettes. Cuban authorities now seem to accept that basic consumer products remain the bread-and-butter of any modern economy. An added benefit: international visitors will find a more ready supply of shampoo!

The Unilever and Brascuba renewals suggest a new realism in the Cuban camp. At ZED Mariel, Cuba is allowing their foreign partners to exert management control, to hire a higher-paid, better motivated workforce, and it is anticipated, to compete in a single currency market. And thanks to the forward-looking diplomacy of Raúl Castro and Barack Obama, international investors are also looking forward to the eventual lifting of U.S. economic sanctions.

This piece was originally published in Cuba Standard.

On Tuesday, October 28, 2014, tens of thousands of citizens of Burkina Faso gathered in its capital city, Ouagadougou, and its second biggest city, Bobo Dioulasso, to protest proposed changes to its constitution regarding term limits. A vote was planned for Thursday, on whether to extend the current limit of two terms to three. This vote is extremely controversial:  Current President Blaise Compaoré, who came to power in a coup in 1987, has ruled the country for 27 years. Allowing him to run for election in November 2015 could extend his reign for another five years. In Ouagadougou on Wednesday, citizens angry over the possibility that parliament might make it possible for Campaoré to stay in power indefinitely set fire to the parliament and forced legislators to postpone the vote that had been set for Thursday, October 30, 2014 to decide the constitutional issue.

The West African country has been plagued by dictators, autocracies and coups in the past. At independence on August 5, 1960, Maurice Yaméogo, leader of the Voltaic Democratic Union (Union démocratique voltaïque), became the country’s first president. Shortly after assuming power, Yaméogo banned all political opposition, forcing mass riots and demonstrations that only came to an end after the military intervened in 1966. Lt. Col. Sangoulé Lamizana and a collection of military elites took control of the government and subsequently dissolved the National Assembly as well as suspended the constitution. Lamizana stayed in power until November 1980 when the military overthrew the government and installed Col. Saye Zerbo as the new president. Two years later, Col. Zerbo’s government was overthrown by Maj. Dr. Jean-Baptiste Ouédraogo and the Council of Popular Salvation (CSP—Conseil du salut du peuple). Although it promised to transition the country to civilian rule and provide a new constitution, the Ouédraogo regime banned all political organizations, including opposition parties. There soon arose a political struggle within the CSP. The radicals, led by Captain Thomas Sankara, eventually overthrew the government in August 1983, and Capt. Sankara emerged as the country’s new leader. In 1984, the Sankara government changed the country’s name from Upper Volta to Burkina Faso and introduced many institutional reforms that effectively aligned the country with Marxist ideals.

On October 15, 1987, Capt. Blaise Compaoré, a former colleague of Sankara’s, killed Sankara and several of his confidants in a successful coup d’état. In 1991, Campaoré was elected president in an election in which only 25 percent of the electorate participated because of a boycott movement organized and carried out by opposition parties. In 1998, he won reelection for another seven-year term. As president, Campaoré reversed all the progressive policies that Sankara had implemented.

In 2000, the country’s post-Cold War 1991 constitution was amended to impose a limit of two five-year consecutive terms on the presidency. However, Campaoré’s supporters argued that because he was in office when the amendments went into effect, they did not apply to him and, hence, he was qualified to run for re-election in 2005. Despite the fact that the opposition fielded several candidates, Campoaré won 80.35 percent of the votes cast in the 2005 presidential election. And, in the presidential elections held in November 2010, he captured 80.2 percent of votes.

Over more than a quarter century in power, Campaoré has used an unusual formula to achieve relative stability in Burkina Faso—authoritarianism mixed with traces of democracy. The complex governance system has relied primarily on Campaoré’s dominant and charismatic political power and has failed to build sustainable institutions—specifically, those capable of maintaining the rule of law and enhancing peaceful coexistence in his absence.

Constitutionally mandated presidential term limits strengthen the rule of law and provide a significant level of stability and predictability to the country’s governance institutions. In response to the efforts by Burkinabé members of parliament to change the constitution to enable Compaoré to secure another term in office, U.S. government officials have recently stated that “democratic institutions are strengthened when established rules are adhered to with consistency.” On his part, Campaoré has proclaimed that his main and immediate concern “is not to build a future for myself—but to see how the future of this country will take shape.” If this is indeed true, then he should exit gracefully from the Burkinabé political scene and henceforth serve as an elder statesman, providing his country’s new leadership with the advice and support that they need to deepen and institutionalize democracy, as well as enhance economic, social, political and human development.

Insisting, as President Campoaré has done, that the constitution be changed so that he can seek an additional term in power not only destroys the country’s fragile stability but also sends the wrong message to citizens about the rule of law—while citizens must be law-abiding, the president does not have to abide by the country’s settled law; if the law stands in the way of the president’s personal ambitions, he can simply change the law to provide him with the wherewithal to achieve those objectives. Such behavior from the country’s chief executive does not augur well for deepening the country’s democracy, an objective that is dear to many Burkinabé. The question to ask President Campoaré is: How do you want history to remember you? As a self-serving political opportunist who used his public position to accumulate personal power and wealth, at the expense of fellow citizens, or as a public servant who led and directed his country’s transformation into a peaceful, safe and productive society?

Israel apologized to Turkey today for the May 2010 incident on board the Mavi Marmara naval vessel, part of a flotilla to Gaza, in which nine Turks were killed from Israel Defense Forces fire. The apology came during a 30-minute telephone conversation between Israeli Prime Minister Benjamin Netanyahu and Turkish Prime Minister Recep Tayyip Erdogan, orchestrated by President Barack Obama, who was ending his 3 day visit to Israel and the Palestinian Authority. Erdogan accepted the Israeli apology, and the leaders agreed to begin a normalization process between Israel and Turkey, following the past three years, when relations were practically at a standstill. (Last December, I wrote about the beginnings of a Turkey-Israeli rapprochement, and discussed more of the policy implications here.)

This development allows the two countries to begin a new phase in their relationship, which has known crisis and tension, but also cooperation and a strong strategic partnership.

The U.S. administration played a key role behind the scenes in creating the conditions that paved the way for an Israeli apology and Turkish acceptance. Undoubtedly, a close relationship between Turkey and Israel-- two of America’s greatest allies in the region-- serves United States’ strategic interests globally and regionally. At a time when the Middle East political landscape is changing rapidly, it was imperative to end the long impasse between Ankara and Jerusalem.

Over the past year, Turkey and Israel have also come to realize that repairing their relationship and re-establishing a dialogue is at their best interest, as they face great challenges in their immediate vicinity (first and foremost, the Syrian civil war).

United States officials emphasized that this is the first step in a long process. Nevertheless, the parties will have to make a great effort to overcome years of distrust and suspicion if they want the relationship to work. No one is under the allusion that relations will go back to what they were in the “honeymoon” period of the 1990s but modest improvement can be made. It will not be an easy task, and for that to happen it is essential that the parties not only talk to each other, but also listen to one another and begin to respect each other’s sensitivities. In order for this rapprochement to be successful, United States will have to continue to oversee discussions between Turkey and Israel, and remain heavily engaged in this process.