Legislating Diplomatic Reform: A Database of Bills
By: Ellice Huang and Sophia Brown-Heidenreich | January 11, 2024
Click here to explore the interactive database of reform legislation in a new window.
Congress has shaped the legal authorities of the Department of State at regular intervals throughout American history, equipping American diplomacy to lead in the face of evolving international challenges. The Foreign Service Act of 1980 was the last major reauthorization of the State Department’s operations and most observers agree that the Department is overdue for its next overhaul.
fp21 created this database of foreign policy legislation to equip US foreign policy reform advocates with a comprehensive resource outlining the historical attempts of Congress to improve the State Department.
Understanding the past is crucial to prepare for the future. This dataset can help us understand what reforms get passed and who advocates for them. It can also help us see what reforms have failed and consider if their time has finally come.
This data set is a first draft – it is not yet comprehensive or exhaustively analyzed. fp21 will continue expanding and improving this database as our work progresses. Computational social science tools can improve our initial natural language processing, resulting in more precise topic tags for faster searching.
Along with our recently released database of DOS reform reports, we hope this database can be an indispensable resource for advocates and policymakers engaged in State Department reform. It offers nuanced insights into the historical dynamics between Congress and State, and serves as a practical guide for navigating the legislative complexities of reform.
With these tools, fp21 will continue its mission to transform U.S. foreign policy to make the best use of evidence, evaluate and learn from policy success and failure, and develop a skilled, diverse, well-trained staff.
This database consists of 471 pieces of legislation relevant to State Department reform introduced between 1900-2023. This includes legislation regarding State Department appropriations and authorizations, foreign assistance, operations and structural organization, and more. The data was collected from congress.gov and the State Department Office of the Historian. For each bill, we collect the date, congress number, sponsor(s), number of cosponsors, a short summary, its official subject, and topic tags.
Of the 471 pieces of legislation introduced, 97 (20%) actually passed to become law. When examining the data by congressional session, we find that the recent 117th and 118th Congresses have been most active in introducing legislation, from 2021 to present. There was a peak in reform legislation activity from 1973-1977, followed by a steady decline around 1993, and finally a steady increase to an all-time peak in the present.
We tagged each bill for certain topics based on whether the legislation contains text related to finance, reform, workforce, or aid. These topic tags were not mutually exclusive and often overlapped. An overwhelming majority of legislation dealt with financial issues: 66% of the introduced legislation and 81% of the passed legislation. The workforce tag was the second most common tag, associated with 25% of the introduced legislation and 13% of the passed legislation.
Note that this dataset is an approximate snapshot of historical State Department reform legislation, and a more refined analysis will be required to improve its accuracy. In addition, data available for 1900-1972 was limited and may be incomplete. More information on the collection and analysis process is available in the appendix and Github (link).
Major Legislation
As a part of the database, we have flagged pieces of legislation that have been identified by the State Department Office of the Historian to have been particularly impactful in the State Department's reform history. These include:
April 5, 1906
Act to provide for the reorganization of the Consular Service of the United States
Reorganized of the consular service, which included increases to pay and reclassifications into nine classes; removed fees again and provided salaries for even the lowest-ranking officers; created a corps of consular inspectors to investigate each consular post every other year
May 24, 1924
Foreign Service Act of 1924 (“Rogers Act”)
Established the Foreign Service, by combining the consular and diplomatic services; created merit-based examinations and FSO rotation system (see fp21’s “Historical Lessons from Reform”)
August 13, 1946
Foreign Service Act of 1946
Created new categories and classes of Foreign Service personnel; refigured assignments, promotion, benefits, retirement systems, and processes; established the “selection-out” system which required the resignation of FSOs who were not promoted within a specific period; required three years of home stay in the first 15 years of an FSO’s service; created foreign service reserve; formally established Foreign Service School into today’s Foreign Service Institute
September 4, 1961
Foreign Assistance Act of 1961
Empowers the president to lead on foreign assistance. (Led to an executive order that established the US Agency for International Development).
October 17, 1980
Foreign Service Act of 1980
Established Senior Foreign Service; divided the Foreign Service into FSO generalists and specialists; increased pay and established a single pay schedule for all FSOs; authorized a foreign service union; ended reserve system of the Foreign Service Act of 1946; strengthened Congressional oversight on human resources matters by requiring regular reports
October 21, 1998
Foreign Affairs Reform and Restructuring Act
Consolidated foreign aid functions within the State Department; abolished ACDA, USIA, IDCA, and reorganized and transferred USAID to the State Department
Notably, the last major legislation flagged was in 1998, 25 years ago. There has been important legislation recently, such as the FY 2020 National Defense Authorization Act signed by President Trump and the National Defense Authorization Act signed by President Biden.
Database Collection Methods
Variables
Each record represents a proposed piece of legislation.
Legislation Number - Each proposed piece of legislation is given a prefix that indicates its form and chamber of origin, and a sequential number. A bill can take one of four forms: bill, joint resolution, concurrent resolution, or simple resolution. Examples include H.R. 1000, S. 500, H.Res. 100. Legislation numbers are not unique and may repeat across congressional sessions.
Congress - Indicates the session of Congress in which the piece of legislation was introduced. The dataset includes legislation of the 82nd - 118th (present) Congress.
Title - The title of the piece of legislation.
Passed - Indicates TRUE if the piece of legislation passed and became law.
Committees - The names of the House or Senate committees associated with the legislation.
Date of Introduction - The date the legislation was introduced to Congress.
Latest Action Date - The date of the latest recorded action associated with the legislation, according to the “Latest Action” data on congress.gov. For passed legislation, this is the date the legislation became public law.
Sponsor - The member of Congress who presented the legislation for consideration.
Number of Cosponsors - The members of Congress who support the piece of legislation.
URL - The URL linking to the congress.gov page associated with the legislation.
Subjects - The subjects associated with the legislation, according to congress.gov.
Summary - The latest summary of the legislation recorded on congress.gov.
Major legislation 1900-2000 - Indicates TRUE if our team has flagged the piece of legislation as particularly important reform legislation, identified from the State Department Office of the Historian administrative timeline. Note that some legislation was not extracted from congress.gov.
Tag_financial - Indicates TRUE if the legislation contains any of the terms appropriate, authorize, fiscal, spend, and fund.
Tag_reform - Indicates TRUE if the legislation included any of the terms modern, improve, reform.
Tag_workforce - Indicates TRUE if the legislation included any of the terms diplomacy, personnel, train, diversity, inclusion, and foreign service.
Tag_aid - Indicates TRUE if the legislation included and of the terms foreign aid, usaid.
Tag_other - Indicates TRUE if the legislation included language associated with miscellaneous terms relevant to DOS reform: statecraft, science, and technology.
Data Collection Procedure and Notes
Data was collected from two sources:
Congress.gov collection of legislation
Metadata on 13,570 bills are downloaded from congress.gov using the search query: [”department of state” OR “state department”] for all available Congresses (1951-2024). Metadata available for download on congress.gov includes: Legislation Number, URL, Congress, Title, Latest Summary, Latest Action taken, Committees, Subjects, Sponsor. Note that available data is limited for legislation introduced before 1972. For more information on legislation process, metadata, and scope of coverage visit the page’s documentation.State Department Office of the Historian
Other landmark legislation between 1900 to 2000 was identified through the State Department Office of the Historian administrative timeline, which lists “principal events concerning the Department of State administration,” such as acts of Congress that established changes in the Department. These are indicated in the dataset with the “Major legislation 1900-2000” variable.
Filtering for relevant legislation
A combination of text tagging and hand coding was used to filter the dataset for relevant legislation. First, “tag” columns were created to categorize each record of legislation. For each of the following categories, a column was created that indicates TRUE if one of the associated terms appeared in the text:
tag_DOS: state department, secretary of state, department of state
tag_financial: appropriate, authorize, fiscal, spend, fund
tag_reform: modern, improve, reform
tag_workforce: diplomacy, personnel, train, diversity, inclusion, foreign service
tag_aid: foreign aid, usaid
tag_other: statecraft, science, tech
We then filtered the database to include only records for which at least one tag column had the value TRUE. The remaining 5,519 records were hand-coded to filter for legislation relevant to DOS reform. This includes legislation regarding appropriations and authorizations, foreign assistance, operations and structural organization, and more.
De-Duplication
Duplicate records with identical legislation titles and congressional sessions (’Title’, ‘Congress’ columns) were merged with the following rules:
Legislation Number, Sponsor: join values by comma
URL, Date of Introduction, Committees, Subjects, Summary: keep first value
Number of Cosponsors, Passed, and all tags: keep maximum value
