By: Thomas Leo Scherer | August 21, 2023

*What are the indicators of foreign policy success?* *Photo credit.*

When is a foreign policy a success, and when is it a failure? With all the op-eds, articles, and reports criticizing foreign policy failures or trumpeting successes, the answer must be obvious, right?

Well, no. Most pieces fail to specify any criteria for success or offer any evidence to support their claim. Rarely will the analysis help us understand why a policy succeeded or failed.

For anyone trying to improve U.S. foreign policy outcomes, this lack of understanding is a serious problem.¹ To identify reforms to improve U.S. foreign policy, we must identify some objective basis for good versus bad policy. The most successful policies should be supported and recreated while failing policies should be altered or aborted. We must use these lessons to train the next generation in replicating successes and avoiding failure.

As with any scientific endeavor, clear standards of success are essential to policy learning, innovation, and improvement. I suggest three rules to follow when evaluating a policy’s success:

Organize policy success into dimensions: product, process, and politics
Delineate alternative policies and how they would have been better
Scrutinize others’ claims of success or failure for ulterior motives

Rule One: Organize policy success into dimensions: product, process, and politics

A key challenge to evaluating policy success is that international affairs are complex. To make sense of that complexity, public policy scholars use heuristics to consider the different dimensions of policy success. One especially useful heuristic is to evaluate a policy’s success based on the 3Ps: products, process, and politics.²

Product success considers the implementation and effects of a policy:

Was the policy implemented as planned?
Did the policy meet its stated goals?
Was the policy an effective use of resources?

Process success refers to how a policy was created:

Was a sufficient, diverse range of policy options considered?
Was sufficient evidence gathered to appraise the policy options?
Was the appraisal of the various policy options reasonable and supported by evidence?
Is there a reasonable explanation for why the chosen policy was selected over the other options?

Politics success considers how a policy is perceived:

Is the policy popular?
Did the policy help the incumbent electorally?
Is the policy well known?

The critical insight from this 3P framework is that policies can succeed in one dimension while failing in another. One study pointed to an example from the HIV epidemic. While France saw a limited number of HIV infections, Spain’s medical authorities were less effective and saw many avoidable infections. France would be considered a product success compared to Spain’s failure. Political success, however, was flipped. In France, the few infections that did happen led to a political scandal with several ministers resigning; in contrast, Spain saw few political consequences.

The 3P framework provides a way to advance the discourse about how exactly a policy succeeded or failed by identifying what dimensions a policy failed in. Each type of success (or failure) calls for a different approach to replicate (or prevent) it in the future.

Rule Two: Delineate alternative policies and how they would have been better

*How do we evaluate the quality of U.S. policymaking in Syria during the Civil War?* *Photo credit*.

Policymaking can be like poker: one will be dealt good hands and bad. Great players won’t win every hand, but they know the odds and will do their best with what they have. In the evaluation of policy success, it is not enough to say someone failed because they lost a hand. Instead, you have to specify an alternate strategy and show that it would have been more likely to succeed.

Consider President Obama’s heavily-criticized decision not to intervene in response to the Syrian government’s continued use of chemical weapons in 2012-2013. The Syrian civil war was undoubtedly a tragedy: hundreds of thousands dead, regional instability, and President Assad still in power. But was there a better option?

Akan Malici makes the case that it was a process success.⁴ President Obama asked his military and intelligence chiefs to develop a range of alternative plans to act in Syria and carefully evaluated the likely outcomes. A few options seemed plausible, most prominently CIA Director David Petraeus’ proposal to arm Syrian rebels, which was supported by Secretaries of Defense and State Leon Panetta and Hillary Clinton. Obama ultimately declined this recommendation and instead put the question of military strikes to Congress. He would later call the notion that arming the rebels would have made a difference “a fantasy”.⁵

But is it possible that arming the Syrian rebels would have achieved a better outcome? Since we cannot observe an alternative reality where this happened, we can never know for sure. Statisticians call this the “fundamental problem of causal inference.”

But we can evaluate the quality of the process by which the decision was made and investigate whether all options considered and accurately appraised. President Obama carefully considered the likely impact of arming the rebels, including asking his team to study how arming rebels had worked in the past. The CIA reviewed past instances and wrote a classified report showing that arming rebels rarely achieved its goals.⁶ Thus, Obama’s request for evidence led to a more accurate appraisal of a policy option, that arming rebels would have likely failed in its goals and would only have made things worse.

Nobody would disagree that the outcome in Syria was reprehensible, but this observation cannot automatically imply that U.S. policy failed. Instead, Malici’s case study presents evidence that Obama considered a diverse range of options, gathered information for an accurate appraisal, and selected the best option based on those appraisals, which makes this a process success.⁷

Malici’s careful analysis of the decision-making process stands in contrast to the many articles evaluating the U.S. response which make unsupported claims about alternative options.⁸ Those articles can still be useful for understanding the political and product results of a policy, and they can influence future choices by making certain costs or benefits more salient. But without showing that there was a process failure and that an alternative policy would have been better, those accusations of policy failure have little to offer for improving U.S. foreign policy.

Rule Three: Scrutinize others’ claims of success or failure for ulterior motives

A key challenge to gauging foreign policy success is that most actors have ulterior motives for saying that a policy succeeded or failed. Policymakers and institutions are rewarded for policy success, so they will want to frame policy outcomes as successful. The audiences will vary – a foreign service officer to their chief of mission, a deputy secretary of state to the congressional hearing, a congressperson to their voters and donors – but in every declaration of success or failure, there is some incentive structure. Even external observers at think tanks or academic institutions exist in systems of incentives. Put simply, “Labeling a policy or decision a ‘mistake’ or ‘failure’ … is an intensely political act.”⁹

For example, I work at fp21, a nonpartisan think tank focused on improving foreign policy processes instead of supporting particular policy or ideological goals. My institution’s identity cushions me against two significant incentive structures – politics and ideology. However, I may still feel professionally motivated to judge that a policy failure is due to a process failure; if policy outcomes aren’t affected by the policy process, I’m out of a job.

Anytime we step into the role of evaluator – whether we are journalists, external experts, Congressional appropriators, or mere citizens – we should be explicit about our own incentives and the motivations of those claiming success or failure.

A More Productive Argument

*Photo Credit PAUL J. RICHARDS/AFP/Getty Images. h/t FP Magazine,* *The Secret to America’s Foreign-Policy Success (and Failure)*.

The shouting matches in the editorial pages over whether to frame policies as successes or failures will undoubtedly continue; today’s media environment tends to reward controversy and audacity over evidence and subtlety. But for those of us earnestly seeking to learn and improve U.S. foreign policy, it is crucial to improve our understanding of how and why a policy succeeded or failed. The three rules listed above will help us get there.

Of the many caveats to consider, I offer two. First, beyond the product, process, and politics framework, there are additional frameworks and heuristics that allow for further nuance in the analysis. Instead of evaluating whether a policy was a success for the American people, for example, one might evaluate the policy from the perspective of another nation or the world at large. We could also be more precise in ‘success-ness’ by using a scale that allows for partial success.¹⁰

Second, providing evidence around policy alternatives is very difficult. Understanding the details that go into a policy decision takes significant research, and in many cases, the information may still be classified. In the Syria case, we only know about the classified CIA review on arming rebels because it was leaked. Decades after the decision to invade Iraq, we still do not have a complete picture of what the State Department was advising, hindering a complete analysis of what went right – or wrong – in that decision process.¹¹ The absence of good information should make us humble in our evaluations, and distrustful of those who shout the loudest.

Nevertheless, policy evaluation remains a critical task for improving the quality of our foreign policy. Improving our collective ability to evaluate the policy decisions will strengthen every stage of the policymaking process. For example, determining who deserves promotion should depend on identifying those who are most capable of achieving success, especially when dealt a bad hand. Further, a clear understanding of success and failure is vital to deliver effective training for future leaders. Ultimately, the efficacy of U.S. foreign policy — and the peace, security, and prosperity of our country — depend on getting this right.

¹ “Specifying the conditions for success or failure of [foreign policy] is arguably one of the most, if not the most, important topic to be studied.” David A. Baldwin, Success and Failure in Foreign Policy

² Adapted from David March and Allan McConnell, Towards a Framework for Establishing Policy Success.

³ Mark Bovens and Paul ‘t Hart, Revisiting the Study of Policy Failures

⁴ Akan Malici, Foreign Policy Mistakes

⁵ Thomas L. Friedman, “Obama on the World”

⁶ Mark Mazzetti, “C.I.A. Study of Covert Aid Fueled Skepticism About Helping Syrian Rebels”

⁷ Instead of the phrase process success, Malici concludes that Obama’s decision was not a ‘foreign policy mistake’.

⁸ Two of the articles I reviewed did provide deeper analysis and evidence. Jeremy Shapiro, who worked at the State Department during the Syria crisis, concluded that the policy failure was that limited assistance was provided instead of no assistance. The BBC article is only certain in the uncertainty, writing “No one knows what would have happened if Obama had decided to arm the rebels in a serious manner…”

⁹ Andreas Kruck, Kai Oppermann, and Alexander Spencer, Political Mistakes and Policy Failures in International Relations

¹⁰ Allan McConnell, Understanding Policy Success

¹¹ Joseph Stieb, “Why Did the United States Invade Iraq? The Debate at 20 Years”