State Department Data Requires Oversight to Avoid Digital Anarchy

By: Dan Spokojny | May 19, 2022

The U.S. withdrawal from Afghanistan was impaired by a lack of good data. Photo by Staff Sgt. Victor Mancilla / U.S. Marine Corps.

There is a lot of excitement to use data to improve operations at the Department of State these days. The State Department’s inaugural Enterprise Data Strategy is potentially a game-changer for the conduct of foreign policy. The Department announced requirements for foreign service officers to build competency in data. A new hiring authority for data scientists is pumping cutting-edge talent into the bureaucracy. And the Department held its first-ever Data for Diplomacy Award, which recognized winners from over 118 global submissions, demonstrating the existing and widespread use of data for diplomacy.   

Progress on data is moving faster than many expected, and credit is due to Secretary Antony Blinken and Deputy Secretary Brian McKeon. Much of this would not have happened without the Foundations for Evidence-based Policymaking Act of 2018, coauthored by former Rep. Paul Ryan (R-WI) and Sen. Patty Murray (D-WA), which obligated agencies to advance data-driven agendas.

Yet the rapid progress on data at State is turning up some pain points within the bureaucracy that require attention.

Data is not being managed well at State. Data scientists struggle to access the information they need in a timely fashion. Tangled and outdated legal authorities tie the hands of analysts trying to collaborate. Valuable data sets collected or purchased by one office are not accessible to other offices. And an archaic IT infrastructure held together with duct tape makes diplomacy’s information superhighway feel more like a dirt road.

Whether a product of turf battles or benign neglect, poor data practices will harm the quality of our foreign policy. The U.S. withdrawal from Afghanistan was impaired by the lack of good data about American citizens in the country. Rather than getting the necessary data to officials immediately, the State Department scrambled to collect and clean the data it needed from dozens of disparate sources from every corner of the government as the withdrawal unfolded. Many of my friends in the foreign service helped clean messy excel spreadsheets with names of potential evacuees. The Washington Post reported that “hundreds of State Department Officials” were enlisted in the crisis response effort.

The demand for data is growing rapidly. Our leaders should act quickly before these challenges proliferate. Fortunately, a few easy fixes exist.

Good Data Practices Will Make Better Foreign Policy

The first step is to give State’s first-ever Chief Data Officer (CDO) oversight over all of State’s data resources. Right now, State cannot account for all of its data and does not possess a central catalog of its resources. Decision-makers lack access to available data reserves, and little incentive or guidance exists to ensure sharing and collaboration. Multiple offices pay for access to the same data sets, wasting taxpayer money and missing opportunities for bulk purchasing or inter-office collaboration.

Data is a strategic asset for the Department and must be treated as such. Assigning data oversight responsibility to the CDO will create a central point of contact for decisions around data management. Research in the private sector finds that companies with an executive responsible for data management achieve superior financial performance. State’s CDO already has some legal authorities, but must be empowered to design solutions that improve capacity for data sharing while ensuring this valuable information remains validated and secure. Some amount of synchronization and standardization of data resources, platforms, and procedures will empower data analysts to better support decision-makers.

Congress already fixed this problem at the Department of Defense. The FY 2020 National Defense Authorization Act endowed the Pentagon’s CDO with “principle responsibility for providing for the availability of common, usable, Defense-wide data sets” and stated that the CDO “shall have access to all Department of Defense data...” The State Department must be granted the same authority to bring governance to the digital anarchy. 

At the same time, Congress must authorize State to procure the data it needs in a timely fashion. Datasets created in universities, nonprofits, and international organizations are often the most carefully constructed and accurate available but cannot be procured through the standard acquisition process. Data that is “made in America'' is not appropriate to purchase with foreign assistance dollars, and it’s inefficient and slow to defer to the regular procurement process for every new dataset made relevant by an urgent policy question. Instead, State should be granted the same authorities as DOD to purchase data under certain thresholds without a competition. At the same time, State should be authorized to obligate grants under the Diplomatic Program account for the purpose of procuring or constructing datasets from universities and academics.

The information technology infrastructure at State is a complex web of outdated systems. The State Department received a “D” grade on information security from a bipartisan report from the Senate, which concluded that “data entrusted to [The Department of State] remains at risk.” Continued investments into the IT Central Fund need to be backed by a strategy to create a safe and secure enterprise data infrastructure. This will allow bureaus establishing their own data products to leverage common software solutions, rather than build new silos.

The good news is that Congress is already thinking about how to address these challenges. The Senate has advanced a bill from Sens. Hagerty and Cardin to create a commission on State Department modernization that would specifically address IT infrastructure, among other issues. And rumors of a “skinny” State Department authorization in the works provide a window of opportunity to address some of these technical issues more quickly.

Policymakers Require the Best Tools

The word “data” can stir a great deal of consternation in some corners of the State Department. Let’s be clear: we are advocating for centralizing the management of data. We do not advocate for centralizing the analysis of that data or removing humans from the analytical process at State. The best analysis in the world is useless if it is mistrusted by decision-makers.

Instead, data analysis should be a tool available to all bureaus and offices in the Department, customized for their unique purposes. For example, the Office of Advanced Analytics in the Bureau of Conflict and Stabilization Operations has developed deep expertise in analyzing political violence. Building out Department-wide capabilities to use the best available evidence in policymaking is an essential step toward building the workforce of the future that Deputy Secretary McKeon describes as a part of the Secretary’s Modernization Agenda. 

Data is a source of national security strength, just like tanks and missiles. The State Department has made great strides by implementing an Enterprise Data Strategy, hiring high-powered data analysts, and more. Congress now has a chance to give diplomats what they need to excel in today’s digital world while making sure valuable data resources are secure.