ISA2023 Roundtable Summary: Can Foreign Policy Decision-Making Be Made More Scientific?
By: Thomas Leo Scherer | April 24, 2023
This article is a summary of a roundtable discussion hosted by fp21 at the International Studies Association annual conference on March 16, 2023, the premier forum for international relations researchers.
Most foreign policy leaders believe foreign policy and diplomacy are an art form.
Ambassador Victoria Nuland, Under Secretary of State for Political Affairs, stated that diplomacy “is not a science” during the State Department’s launch of its Learning Agenda in 2022. Ambassador Barbara Bodine, former Dean of the School of Professional Studies at the Foreign Service Institute, agreed; “It’s almost impossible to quantify what we do, and in fact, I think that there’s a great danger in trying to quantify it.”
These views are a challenge for scholars who believe their research should inform policymaking. Scientific research is of no use to leaders who are predisposed to categorically devalue it.
Against this backdrop, the participants in our roundtable—leaders from government, academia, and think tanks—discussed a timely question with potentially profound implications for U.S. national security: Can foreign policy decision-making be more scientific? What would that look like? What are the challenges and risks? Most agreed that decision-making indeed can and should be made more scientific, though there were important details regarding what such changes entail. Overall, participants agreed that US Foreign policy will be more effective if we continue to challenge the dominant culture of instinct-driven decision-making with an alternative culture of evidence-driven decision-making.
Less Instinct, More Evidence
Dan Spokojny, CEO of fp21, saw the anti-science sentiment in foreign policy as an epistemological problem. In many fields—finance, public health, education, psychology, medicine, engineering, etc.—knowledge is accumulated by collecting data and testing hypotheses. In contrast, knowledge in foreign policy is based on instinct and judgment developed through experience and tenure.
Dr. Alex Bollfrass, senior researcher at the Center for Security Studies at ETH Zürich, called for foreign policy to create its own “culture of evidence” as exists in journalism, law, and academia. In law, each side shares the best evidence for their position, but they are not allowed to hide evidence that would favor the other side. Reputable journalism follows the ‘two-source’ rule before presenting something as factual. Academia’s systems of training, evaluation, and peer review build and protect its culture of evidence. Most fields maintain their own credential and entry processes, such as the bar exam or medical licenses.
Bollfrass explained that foreign policy uniquely lacks a clear culture of evidence. New practitioners arriving from different backgrounds receive little guidance on how evidence should be handled and evaluated and no formal training on how to make decisions based on evidence. Foreign policy needs its own culture of evidence that is tailored to the needs and constraints of the field.
Increasing Demand for Science
There is already some momentum toward more science in foreign policy. Dr. Jessica Lieberman, Deputy Managing Director of the State Department Office of Foreign Assistance, said that two recent laws—the 2018 Evidence Act and the 2016 Foreign Aid Transparency Act—have significantly increased the expectations for evidence use in foreign policy. The State Department recently launched the Learning Agenda—a four-year plan to answer eight policy-relevant questions, and the Office of Foreign Assistance is actively setting up research partnerships as part of that effort.
Nonetheless, Lieberman sees significant barriers to evidence at the State Department: lack of staff capacity, poor knowledge management, and insufficient time. She believes the State Department needs social scientists—anthropologists, sociologists, political scientists, and more—to help elevate a culture of learning and evidence within the State Department.
Professor Jordan Tama at American University sees a demand for scholarly input in government, but the lack of an institutional process to get that input means it happens in an improvised and ad-hoc manner. Tama co-directs the Bridging the Gap project that meets that demand by helping scholars meet policymakers where they are. Their New Voices in National Security initiative has facilitated workshops where scholars prepare answers to pressing policy questions, most recently on emerging technology security challenges and on what the Ukraine conflict tells us about sanctions.
To support evidence-based foreign policymaking, Dr. Matt Malis, incoming professor at Texas A&M, advised social scientists to pursue a problem-driven research agenda and to focus on causal mechanisms amenable to intervention. While macro-level factors such as economic development and regime type may be major factors for explaining foreign affairs, factors like diplomatic visits and military aid are actionable and thus more useful to policymakers.
Building a Culture of Evidence
Dr. Rachel George, Council on Foreign Relations Education Director and Duke University Lecturing Fellow, highlighted three core tenets to an institution’s learning capacity: an appetite for risk and failure, incentives for the necessary skill sets in the workforce, and strong frameworks to monitor and evaluate (M&E) their work.
These are exemplified in UK’s “What Works” project on preventing gender-based violence. This project demonstrated an appetite for risk but leveraged robust M&E frameworks. A diverse team of researchers found, for example, that economic programs to increase stability and empower women in Uganda were having unintended effects; while economists found that the programs indeed empowered women financially, anthropologists found that they increased the risk of domestic violence.
Not all evidence leads to learning. Professor Susanna Campbell from American University, Director of the Research on International Policy Implementation Lab explained that learning is not just the intake of information but rather is what is done with that information to get an organization closer to achieving its goals. Policymakers require specialized knowledge to make sense of the evidence their organization produces; a doctor can do more with a blood test than a lawyer can.
Learning also requires decision-makers to be engaged with the problem and open to change. Campbell’s research finds that learning is most likely when the decision-makers get close to the problem and can see and feel the evidence. To be most effective, decision-making needs to be on the front lines where the problem exists.
An attendee asked whether we should test the effects of a more scientific decision-making process. Several panelists agreed that continually evaluating the impact of any given policy or reform is a vital aspect of evidence-based policymaking. In this respect, scientific decision-making must be self-reflective.
But this question raises the challenge of evaluating foreign policy success and failure. I described three ways to assess policies. First, policies are labeled as success or failure after the fact, often with political motives. Second, policies can be evaluated with preset criteria, such as whether the policy achieved its stated goals. Finally, a policy can be evaluated on whether its formation followed some ideal process, regardless of the outcome achieved. While we at fp21 are most interested in improving and increasing process success, we should understand that policymakers may be more interested in achieving political success.
Foreign Policymaking Can Be More Scientific, and so Can Science
The event concluded with the audience asking participants whether foreign policymaking could and should be more scientific?” We largely agreed that it could and should be, but any reforms should also be based on evidence and subsequently evaluated. Bollfrass reminded us while foreign policy can be more scientific, we are under no illusions that science itself is ever perfect. “Science itself can be made more scientific! Foreign policy can also be more scientific by taking advantage of the rich methodological and conceptual toolkit that science has developed.”
