Improving Data Utilization in Diplomacy
This piece was originally published by Stimson Center in a collection called State Department Reform Under the Second Trump Administration
By Lula Chen | March 6, 2025
The State Department’s modernization agenda included many commendable ways to improve the department’s use of data. The Enterprise Data Strategy aimed to transform how the State Department collected and managed its data, creating a culture of using data to enhance decision-making and diplomacy. Across many metrics, the Department has also largely been hitting its data-informed diplomacy goals, providing training, hiring data scientists and data officers, producing artificial intelligence (AI) use cases, and increasing the number of data assets. These changes help create a foundation that improves data utilization at the State Department and should be continued.
Even with new training, assets, and staff, improvements in both data collection and data practices can still be made. Below are four suggestions to further improve the use of data and data integration into decision-making at the State Department in the Trump administration. These suggestions are not meant to be comprehensive but point at ways to strengthen data utilization moving forward.
Identify key decisions where evidence from data is valuable. Data is most effectively used when it is connected to decisions. Before considering how to use data, policymakers should determine the key decisions that would benefit from data-supported evidence. For example, there may be certain decisions where two or three options seem viable, and data can help clarify which of the options aligns most closely to strategic goals. Identifying the decisions that will benefit from data will help to focus the data collection and ensure its utility for the new administration.
Identify a few, high-quality metrics and indicators that provide compelling evidence for a decision. Compelling evidence is often available through a few key, high-quality pieces of data. Data quality dimensions, such as accuracy, completeness, consistency, and timeliness, help to ensure that the data are reliable for decision-making. The State Department has identified improving data quality as part of its Enterprise Data Strategy, and the new administration can continue to focus on improving the data quality of the department.
High-quality data can be costly; however, having a few, critical, high-quality indicators may be much more useful than having many low-quality, noisy, indicators. Right now, bureaus often tend to collect numerous metrics, as seen in Integrated Country Strategies or programmatic monitoring and evaluation plans, and the quality of those metrics vary widely. An exercise to focus the metrics into a few key indicators collected with quality in mind, including publicly available indicators that have been rigorously validated, will make data collection and use much more relevant for decision-making in the new administration. Good indicators must accurately capture the concept they are measuring. To assess the overall validity of indicators, researchers consider content validity, convergent/discriminant validity, and nomological validity.
Content validity means that a measure includes all relevant elements of a concept and excludes irrelevant elements. Convergent/discriminant validity means a measure correlates statistically with similar concepts but not with dissimilar ones. Nomological validity means a measure is predictive of other things that the measure is expected to affect. This USAID resource and this article provide more information on assessing the validity of indicators. Adopting practices for assessing the validity of indicators can help the new administration ensure that they are sound and relevant for decision-making.
Lower barriers to data access. There is already a lot of data at the State Department; however, their accessibility may be limited. The Department has hit its targets for the number of data assets it has available, but it is unclear the extent to which they are being used. The limited use of these data assets may be due to accessibility barriers, such as a lack of system interoperability and data silos, the preprocessing needed to make data analyzable, and a shortage of technical personnel.
Numerous databases and systems throughout the State Department are not interoperable and do not communicate with each other. In addition, different bureaus have access to different data, and sharing data between bureaus may be difficult due to privacy concerns or other issues. Both situations make it difficult to merge or integrate data from different sources together, which limits the usefulness of data. Another barrier is the form of data assets. Many of the data assets at the State Department require several layers of additional work before the data can be used. For example, PDFs and documents provide a wealth of data, but those documents must be preprocessed, and the data they contain must be extracted and cleaned, before they can be analyzed. Lastly, there is still a need for more data scientists, especially those with subject area expertise, to be available to help bureaus with their data analysis needs. The new administration can lower barriers to data use by increasing interoperability between systems and data accessibility across bureaus, making data assets analysis ready, and providing more technical capacity to bureaus.
Relatedly, AI and machine learning tools are integral for data scientists analyzing data and for those who want to derive actionable data-based recommendations. Tools like NorthStar summarize news stories from over 100 languages, reducing the workload of diplomats. Trainings, tools, libraries, and use cases are available through an AI hub at the State Department, and more tools are becoming available with initiatives like ChatGPT Gov. The new administration should continue integrating AI into the workflow of the State Department, with cautions around when these tools may be biased or hallucinate, and reminders of the practical and ethical boundaries of what these tools can or cannot do.
Interpret data with subject matter experts. Good data utilization requires sound interpretation of the data. Making accurate interpretations requires contextual knowledge and understanding of what the data means for a particular situation. The new administration can create processes that bring together groups of in-house and external experts, including foreign service officers and civil servants, throughout the data and analysis process to improve the interpretation and use of data.
The Enterprise Data Strategy targeted a culture change at the State Department, where evidence from data is routinely part of the decision-making process. Building on existing foundations, these suggestions offer ways to make integrating data use with decision-making more feasible and easier to practice. Adopting these suggestions can enhance the effectiveness of future programs.