Government Surveillance and Declassified Documents | Reference eReviews

THOSE OF US WHO can bear to watch the news have learned much more about the intelligence-gathering capabilities of both governments and nongovernmental entities over the course of the last year than we probably care to know. What the resources examined here tell us is that governments have been in this business for a very long time.

The Digital National Security Archive (from ProQuest, working with the National Security Archive) includes 50 curated collections of declassified documents relating to U.S. foreign policy, human rights, nuclear history, domestic surveillance, and the intelligence community itself dating back to the Harry Truman administration, while U.S. Declassified Documents Online (Gale) reveals once-secret presidential documents that go as far back as 1900.

The formerly classified files in ProQuest History Vault: Black Freedom Struggle show how the march toward civil rights produced so much fear within the federal government that it felt compelled to use the intelligence apparatus against U.S. citizens. Secret Files from World Wars to Cold War (Routledge/Taylor & Francis) document formerly secret intelligence activities and their role in British foreign policy from 1873 to 1951.

Digital National Security Archive
ProQuest;
http://www.proquest.com/products-services/databases/dnsa.html

Free trial available

CONTENT The Digital National Security Archive (DNSA) represents a collaboration between ProQuest, in the role of curators of the collections, and the National Security Archive, an independent nongovernmental research institute located at George Washington University.

The National Security Archive—relying on the expertise of former government officials, scholars, and journalists—has obtained some 93,000 declassified documents largely through its use of the Freedom of Information Act (FOIA). Additionally, there are more than 150 audio files associated with former secretary of state Henry Kissinger’s impact in U.S. policy during the Nixon administration.

In its DNSA incarnation, thanks to ProQuest’s efforts, this material has been organized into coherent topical collections around a specific event or policy issue. Each item is cataloged, abstracted, and indexed, and each collection comes with an array of research aids including an introductory essay, which provides context and vital background information prepared by experts in the intelligence and foreign policy arenas; a chronology; a bibliography; and a glossary.

The range of collections traces the often dark narrative of the past seven decades. About a quarter of them deal with U.S. policy around the globe. Drugs and human rights, often entangled, form the theme of several collections, and crises—in Berlin and Cuba—and scandals such as Iran-Contra are documented as well. Several collections are devoted to themes such as espionage and intelligence, terrorism, weapons of mass destruction, electronic surveillance and national security, nuclear history, and the intelligence community. And many others—the military uses of space, the CIA’s domestic intelligence program, Iraqgate, and Edward Snowden—would make compelling reading for a wide spectrum of scholars and researchers.

A number of collections have been updated since their original release with newly declassified material or a more expansive range of dates. The Cuban Missile Crisis collection, updated twice, now includes documents from the Soviet archives as well as the official, multivolume CIA history of the disastrous Bay of Pigs operation, complete with a slash through the “top secret” classification on the title page.

DNSA’s content runs the gamut of primary source material—policy documents, presidential directives, White House communications, memos, diplomatic dispatches, notes on meetings, independent reports, briefing papers, email, confidential letters, handbooks and manuals, legal opinions, talking points, white papers, and other items.

The collections date from the Truman presidency (U.S. Nuclear Non-Proliferation Policy, 1945–1991) through to the Obama administration (Mexico-United States Counternarcotics Policy, 1969–2013 and Cuba and the U.S.: The Declassified History of Negotiations To Normalize Relations, 1959–2016).

Counting original collections plus a growing number of updates, DNSA currently totals more than 50 topical collections, with ProQuest adding roughly two new ones per year.

USABILITY There are a standard set of options for getting started: basic and advanced search modes, browsing capabilities (including bibliographies, chronologies, and glossaries), and the “about” section, which briefly describes current, new, and forthcoming collections. A basic search box is centered on the screen, and the list of individual DNSA collections sits below that.

Mousing over the “information” icon for each collection pulls up a 100-word description of its contents—number, type of records, and source—plus sufficient political and historical context to induce interested researchers to dive in. The annotation for CIA Covert Operations II: The Year of Intelligence, 1975, for example, states that the collection details covert operations that took place during the Henry Ford administration and media revelations about those official abuses, which ultimately led to the reorganization of the U.S. intelligence service. When users click “more information,” they’re taken to a “scope and sources of the document set.” From there, researchers can navigate to the essay (although not all of the newer collections have one) or the names and subject lists.

We searched for “Seymour Hersh,” the investigative journalist who dedicated his career to exposing issues such as the My Lai Massacre during the Vietnam War, across all collections and got 97 hits. Opening up the DNSA collection filter revealed that 28 documents came from the various Kissinger collections. The transcripts of conversations (usually off the record) between Hersh and the former secretary of state provide insight into a journalist’s process and the sometimes critical response to Hersh’s articles, with Kissinger at one point complaining, “There won’t be a foreign policy if this keeps up.”

Selecting the Nicaragua: The Making of U.S. Policy, 1978–1990 collection, we did an advance search of “contras” (the right-wing counterrevolutionaries). Browsing through the “document type” index and checking “manual” yielded just one hit, but it was a fascinating item—the illustrated Freedom Fighter’s Manual, in English and Spanish. This CIA sabotage text aimed at patriotic Nicaraguans in their struggle against the leftist Sandinistas suggests ways to paralyze the Marxist state, ranging from being late for work to wasting energy by leaving the lights on to threatening one’s boss.

Interestingly, a Wikileaks search, filtered by DNSA record type “document,” revealed that 692 files had been obtained from WikiLeaks.

Results may be sorted by date and marked for emailing, printing, downloading, or saving. Boolean operators, truncation, and wildcards are all available. Some, but not all, documents allow users to search by keyword through a downloaded PDF.

Unfortunately, DNSA doesn’t feature a general browse option. Researchers don’t always know what they want—or what’s important—until they see it, and this function (in ProQuest History Vault, for instance) would allow them to scan through the files until they turn up something eye-catching.

PRICING ProQuest pricing is based on full-time enrollment (FTE), population served, existing ProQuest database holdings, and consortium agreements.

VERDICT DNSA is one of the best ways to look beyond the textbook versions of U.S. history and foreign policy and understand the heroic, cynical, brutal, visionary, and unintentionally comic elements that all play a role in how policy is made, communicated, implemented, and, in many cases, hidden. With appeal beyond the regular confines of academic disciplines, DNSA is intrinsically fascinating.

ProQuest History Vault: Black Freedom Struggle in the 20th Century
ProQuest;
http://www.proquest.com/products-services/historyvault.html

CONTENT ProQuest History Vault collections offer researchers access to key primary source material in the form of digitized letters, reports, papers, photographs and illustrations, scrapbooks, financial records, diaries, news clippings, press releases, speeches and addresses, and other types of printed materials, including advertisements, brochures, and pamphlets. The material is organized into thematic modules, and coverage across the entire platform includes civil rights, Southern life, slavery and the Civil War, American politics and society, international relations and military conflicts, women’s studies, and workers and labor unions.

ProQuest History Vault: Black Freedom Struggle focuses on African Americans’ fight for liberty and basic rights. The resource consists of four modules—two pertaining to federal government records, and two to organizational records and personal papers—offering unique documentation and a variety of perspectives. A partial inventory of titles includes FBI Files on Black Extremist Organizations, Part 1: COINTELPRO and the Deacons for Defense, and Part 2: Huey Newton and Eldridge Cleaver of the Black Panther Party; The Martin Luther King, Jr. FBI File, Part 1, and Part 2: The King-Levison File; Federal Surveillance of Afro-Americans, 1917–1925, The First World War, the Red Scare, and the Garvey Movement; Department of Justice Classified Subject Files on Civil Rights, 1914–1949; and Records of Student Nonviolent Coordinating Committee (SNCC) and The Congress of Racial Equality (CORE).

Files detail race riots of the 1920s; lynching investigations; press reports and NAACP meeting records; speeches, essays, interviews, and articles about Huey Newton and the Black Panther Party; Freedom of Information Act records on busing, school desegregation, and fair housing; Justice Department special reports on “Negro activities”; and verbatim transcripts and detailed summaries of telephone conversations.

Additional History Vault modules that comprise formerly classified material include FBI Confidential Files and Radical Politics in the U.S., 1945–1972; Confidential U.S. State Department Central Files, 1960–1969; and U.S. Military Intelligence Reports, 1911–1944. A comprehensive list of collections is available on the History Vault website.

USABILITY Institutions may subscribe to modules and collections in just one subject area or from a combination of different subjects, with search options limited to one module or extended to numerous ones. With such varied documents and subject matter, we found the best way to begin was to “browse collection,” either by module or by collection title. Each collection includes a detailed description of the contents, offering researchers background information and possible keywords. A “view all documents” tab also presents a starting place.

With comprehensive metadata associated with each file, researchers can drill down from the title to view linked item-level information (subject, federal agency, organizations, etc.) as well as collection-level data (topics, source, and more.)

Starting with a broad keyword search for “Kennedy,” we narrowed results by federal agency (FBI) and by subject (electronic surveillance) to retrieve 227 records. The first few in the list were newspaper clippings collected by the FBI, all pertaining to Martin Luther King Jr. Browsing through several of the folders, we read numerous accounts of King’s stabbing, his arrest on income tax charges and possible perjury charges, and the “spreading South-wide” series of sit-down protests at segregated lunch counters throughout four states. Another file included numerous letters of support and flattery written by President John F. Kennedy to FBI director J. Edgar Hoover, which praised Hoover’s vocal opposition to King having received the Nobel Peace Prize.

A query for “Cleaver” and “speech” within the FBI Files on Black Extremist Organizations, Part 2: Huey Newton and Eldridge Cleaver of the Black Panther Party retrieved 51 records. An internal FBI memorandum reported that Eldridge Cleaver “used obscenities in attacking presidential candidates Richard Nixon, Hubert Humphrey and George Wallace” during his speech at the “Black America: Evolution or Revolution” symposium at the University of California Los Angeles.

Users can download the entire file or specific sections—and the system displays the number of matches based on the search terms in each section. A “search within” the results option also helps with specificity. This is somewhat helpful when dealing with large documents, but users must still page through to find the matches, which we were happy to do to discover the variety of content and subject matter.

Other options besides “browse collections” include “basic” and “advanced search” and “timeline browse.”

PRICING Cost varies according to a number of factors, including the type of library or population served. Please contact your ProQuest representative for more information.

VERDICT Through special FBI reports on suspected communist activities, firsthand accounts of domestic surveillance, and the powerful metadata, this resource provides access to untapped materials and offers background on the political and social climate of the 20th century, delivering more than textbooks and other secondary sources supply. Black Freedom Struggle will complement coursework across disciplines.

Secret Files from World Wars to Cold War: Intelligence, Strategy, and Diplomacy

Routledge/Taylor & Francis;

www.secretintelligencefiles.com/

Free trials available

CONTENT Secret Files from World Wars to Cold Wars (LJ 9/15/17) provides access to nearly 17,000 British government secret intelligence and foreign policy documents from 1873 to 1953, with the majority dating from the 1930s and 1940s. ­Although World War II is central to this resource, the database also covers the Spanish Civil War (1936–39), the early years of the Cold War (1947–91), and the Korean War (1950–53), helping place intelligence and foreign policy decisions in the context of international activities leading up to and following World War II. Organized within ten themes, such as “British Domestic Security,” “Foreign Policy and International Relations,” “Military Intelligence and Operations,” “Propaganda, Censorship and Psychological Warfare,” and “Signals Intelligence and Code-Breaking,” the 4,500 files originate from the UK National Archives.

The documents were selected through consultation with Routledge/Taylor & Francis Group’s editorial board, whose members specialize in 20th-century intelligence, foreign policy, and international relations. The goal of the archive is to provide research into the “intersection of secret intelligence with foreign policy, military strategy, and international relations during the first half of the twentieth century.” Materials include correspondence, directives, financial records, maps, meeting minutes, memoranda, and signal intelligence reports.

Nine series comprise Secret Files, with the majority of content derived from the Foreign Office and the Permanent Under-Secretary’s Department (PUSD) files. The PUSD records—categorized as FO1093—concern operational intelligence-related matters and foreign and defense policy issues, as well as information collected and received by covert means. Other series include Joint Intelligence Sub-Committee: Minutes and Memoranda, 1936–1939; War Cabinet and Cabinet: Committees and Sub-committees of the Chiefs of Staff Committee: Minutes and Papers, 1939–1947; and Government Code and Cypher School: Signals Intelligence Passed to the Prime Minister, Messages and Correspondence, 1940–1945.

Documents worth noting include Adolf Hitler’s “Last Will and Testament,” with official memoranda debating the implications of when to release (or suppress) the information to the general public; the Rudolf Hess Files, with translated copies of letters from Hess to Hitler and to his family and friends in Germany; information on signals intercepted and deciphered by the British Government Code and Cypher School at Bletchley Park during World War II; and “Operation Embarrass,” postwar activities by the British Secret Intelligence Service (SIS) that examined possible measures to reduce Jewish migration to Palestine.

A series of essays written by the editorial board contextualize the material and highlight key themes, topics, and events included within the resource.

USABILITY Key functionality of Secret Files includes optical character recongition (OCR)–enabled documents with browsing and filter options by series, conflict, theme, time period, regions, document types, and organization. Users can begin with a “quick search” at the top of the screen or browse via “explore content by theme” and “what people are searching for” (a list of popular queries that at the time of review included “Ian Fleming” and “Adolf Hitler”).

A resources box, with links to FAQs, thematic essays, video tutorials, and detailed series descriptions, can also be found on the home page. An “about” tab at the top navigation menu links to a site overview that covers key people (politicians, royalty, important committee members), key organizations, methodology, and more.

Browsing the theme “Weapons Technology and Nuclear Warfare” retrieved 516 results, and applying filters for more precision, we selected “Second World War,” “Europe,” and “United Nations.” Refining produced a better result set, including a report discussing Japanese capabilities and intentions to wage gas warfare, as well as the document “Government Code and Cypher School: Signals Intelligence Passed to the Prime Minister, Messages and Correspondence,” “a file of signals, intelligence reports, messages, and correspondence issued by the Government Code and Cypher School and sent by the head of the Secret Intelligence Service (MI6) to the Prime Minister, Winston Churchill.”

Advanced search allows for the combination of terms and filters, helpful when pursuing an actual research topic. The query “Hitler,” when limited to “Second World War” and “maps,” retrieved 16 documents, including “Ability of German Army in the West To Continue the War and Prolongation of Nazi Resistance after Loss of Berlin and Northern Germany.” The document type “maps” is somewhat misleading, as the category also includes reports and memoranda. Each document displays a “keywords in context” tab, which allows users to expand and view the most relevant matches in the document or section.

The full record displays extensive metadata, which is also helpful for additional search terms, a description of the document, numerous keywords, and the option to “view images” or download the entire document as PDF. A sidebar displays related content for each document selected—typically a list of ten to 12 additional resources. From the search for “Hitler,” selections included a report on the German strategy and capacity to resist, information on the German situation after the capture of Berlin and loss of the Ruhr area, and translations of personal letters from Rudolf Hess to his wife, Ilse Hess; his loyal “adjutant” Karlheinz Pintsch; his parents; and others.

PRICING While Taylor & Francis digital resources vary in cost, outright purchase fees for perpetual access range from $25,000 to $75,000, based on an institution’s FTE and research output. Trials available.

VERDICT With a powerful interface and all the right bells and whistles for serious researchers, there’s no question this resource delivers the goods, providing an in-depth view of World War II from a rare, high-level government perspective. Although those less familiar with the topic will appreciate the presentation of material that illuminates in detail pre- and postwar activities, scholars will most likely benefit the most from this rich collection.

U.S. Declassified Documents Online

Gale Cengage Learning;
https://www.gale.com/c/us-declassified-documents-online

Free trial available

CONTENT U.S. Declassified Documents Online (USDDO) provides scholars, researchers, and inquisitive citizens with access to previously classified federal records produced over the last 120 years. It represents, according to information on the USDDO homepage, “the most comprehensive compilation of declassified documents from the executive branch,” including intelligence studies, policy papers, diplomatic correspondence, cabinet meeting minutes, briefing materials, and domestic surveillance and military reports.”

USDDO’s editors keep a close eye on the various presidential libraries for the release of newly declassified documents to add to the collection. While presidential documents make up the bulk of the material, previously classified documents from the U.S. State Department, the Department of Defense, the CIA, the FBI, the National Security Council, and other executive agencies supplement the presidential papers and round out the collection.

There’s a lot of history in these files—from the run-up to World War I through the ongoing War on Terrorism—and, as the “about” section notes, USDDO “compiles the declassified documents, released individually or in sets, that fill in the most sensitive gaps of the historical record left by the federal government.”

The original back file is comprised of more than 100,000 documents and 700,000 page images, with document dates ranging from 1900 to 2008. Two subsequent updates add about 9,500 items to the total and feature documents with publication dates through 2017, including material as recent as former director of defense Michael Flynn’s resignation letter. While the vast majority of the collection consists of previously classified information, not everything here was once secret.

The “about” section features a valuable tutorial on how the federal government handles classified information, the levels of classification, the threats posed when secrecy is breached, and the unpredictable process by which such documents are declassified.

USABILITY With the dissemination of another batch of declassified documents just in the news, we entered “Kennedy assassination” into the prominent search box on the home page, producing a results list of 299 monographs (i.e., nonserial publications). Each item displayed a thumbnail of the document, source, date, number of pages, and links to keywords in context and full citation. The document type was displayed to the right, and a lengthy list of filters was shown on the left side of the screen. Searchers may analyze results using the term clusters and term frequency options. We clicked “term clusters,” which displayed words and subjects found most often in the text of the search results, arrayed in a wheel or in tiles. John F. Kennedy’s name, on the inner ring of the wheel image, was connected to, among other terms, the name Jack Ruby on the outer ring. Clicking on Ruby brought up a list of 11 documents, including one from the Washington Star in which Watergate burglar Frank Sturgis claimed that Ruby met with Fidel Castro in Cuba ten weeks before the Kennedy assassination to discuss drug smuggling, arms purchases, and “the removal of the president.”

Advanced search lets users filter by keyword, document title, place of publication, subject, entire document, or document number. The “allow variations” check box permits searchers to retrieve documents where there are spelling variations.

USDDO limiters include publication date, declassified date, document type, classification level, sanitization (sanitized documents are those in which text has been deleted, crossed out, or obscured in some way by government censors), completeness, and source library.

Users can run searches without typing in any terms—that is, relying on just the limits. Using the declassified date limiter, we determined that 24 declassified documents had been added to the collection since January 1, 2017, half of them relating to Hillary Clinton.

Using Gale’s Image Viewer, searchers can zoom in, making documents full-screen size, and adjust brightness and contrast.

Documents can be bookmarked, downloaded (including with a full OCR text option), shared, printed, and emailed, and the citation tools permit formatting results in MLA 8th, APA 6th, and Chicago 16th edition formats. Additionally, registered users are able to save documents to their personal folders and add public or private tags.

PRICING Fees vary depending on population served. Contact your Gale representative for a quote.

VERDICT U.S. Declassified Documents Online makes accessing formerly classified material fairly simple and will appeal to researchers of those aspects of U.S. policy, security, and history that have kept secret by every administration over the course of the 20th and 21st centuries.

E-SHORT TAKES

Archives Unbound Series: Federal Response to Radicalism in the 1960s; Federal Surveillance of African Americans, 1920–1984; America in Protest: Records of Anti-Vietnam War Organizations—Vietnam Veterans Against the War
Gale Cengage Learning;
https://www.gale.com/primary-sources/archives-unbound

Archives Unbound presents topically focused collections of historical documents. Organized alphabetically by group, Federal Response to Radicalism in the 1960s covers a wide range of viewpoints on political, social, cultural, and economic issues. Documents include COINTELPRO: The Counterintelligence Program of the FBI, covering the Communist Party of the USA, hate groups, the Socialist Workers Party, and Cuban groups supporting Fidel Castro. Also included are FBI files on people, groups, and events considered a threat to national security, such as Abbi Hoffman, the Black Panther Party, Cesar Chavez and the United Farm Workers, and Malcolm X. The files highlight surveillance, investigative and legal reports, interviews with students and guardsmen, maps, contemporary news accounts, speeches, digests of FBI phone taps, handwritten letters, and more.

Illustrating the infringement of First Amendment freedoms by the FBI, Federal Surveillance of African Americans, 1920-1984 covers the origins of the black social protest and separatist movement and individuals and organizations with suspected Communist ties.

Many of the documents in this collection are based on accounts provided by black “confidential special informants” enlisted by the FBI to infiltrate and discredit groups such as the Nation of Islam. Documents include COINTELPRO: Black Nationalist “Hate” Groups and FBI reports on events, individuals, and organizations deemed politically suspect such as Adam Clayton Powell, the Atlanta Child Murders, Marcus Garvey, Malcolm X, Thurgood Marshall, the Reverend Jesse Jackson, the National Association for the Advancement of Colored People (NAACP), and the Committee for Public Justice. Among the documents are memos, lab tests, detailed trial accounts and records, speeches, and news clippings.

America in Protest: Records of Anti-Vietnam War Organizations, the Vietnam Veterans Against the War (VVAW) covers roughly the activities of the VVAW from 1967 to 1976. The group began with a few Vietnam veterans protesting for peace in New York City in 1967 and continues today, working for peace, justice, and the rights of veterans. This collection consists of FBI reports covering both VVAW and other organizations, such as student groups and Communist groups. There’s also an FBI historical overview on leading antiwar organizations during the Vietnam War era.

The resource features in-depth reports, a copy of the New York Times advertisement “Vietnam Veterans Speak Out,” and memos related to both the advertisement and the demonstration at New York City’s Union Square. The database also covers legal cases involving constitutional rights, wiretapping, and civil rights; surveillance memos; letters from and to J. Edgar Hoover; VVAW newsletters and flyers; position papers by leading antiwar movement activists; detailed surveillance from informants on college and university campuses; and documentation on the Kent State shootings and subsequent campus strikes. Surveillance reports on well-known activists such as Jane Fonda, John Kerry, and Howard Zinn are also included.

Users can also search across all Archives Unbound purchased collections and limit by category or collection title.

Confidential Print: Africa, 1834–1966; Latin America, 1833–1968; Middle East, 1839–1969; North America, 1824–1961

Adam Matthew Digital;
http://www.archivesdirect.amdigital.co.uk/

The “Confidential Print” series, issued by the British Government, covers more than 100 years of political, social, and economic history pertaining to countries of the Arabian Peninsula, Turkey, Iran, Afghanistan, Egypt, and Sudan. Per the database, “All items marked ‘Confidential Print’ were printed and circulated immediately to leading officials in the Foreign Office, to the Cabinet, and to Heads of British missions abroad.”

With full-text search and top-level metadata, this series includes complete volumes of all the Foreign and Colonial Offices publications, with content such as diplomatic correspondence, letters, telegrams, reports, surveys, material from newspapers, statistical analyses, annual reports and calendars of events by country, published pamphlets, ephemera, military papers, profiles of prominent individuals, maps, and texts of treaties. Subject coverage across the collections illustrates the resource’s diverse range.

Researchers can select from a list of popular search terms as a starting point, such as the 19th-century Egyptian reforms of Muhammad Ali Pasha, the Suez Crisis of 1956, the Partition of Palestine, the Arab-Israeli Conflict, Prohibition, reports on the Ku Klux Klan, Nazi activities in Chile, the Atoms for Peace conference, border disputes, British claims on El Salvador, Panama’s independence from Colombia, sugar and slavery in Cuba, and more.

This resource is available on the Archives Direct platform and is cross-searchable with other Adam Matthew Digital collections within the portal, affording seamless browsing across all content purchased by an institution.

Gail Golderman (goldermg@union.edu) is Electronic Resources Librarian, and Bruce Connolly (connollb@union.edu) is Reference & Bibliographic Instruction Librarian, Schaffer Library, Union College, Schenectady, NY

This article was published in Library Journal. Subscribe today and save up to 35% off the regular subscription rate.

Share
About Bruce Connolly & Gail Golderman

Gail Golderman (goldermg@union.edu) is Electronic Resources Librarian and Bruce Connolly (connollb@union.edu) is Reference & Bibliographic Instruction Librarian, Schaffer Library, Union College, Schenectady, NY

Comment Policy:
  1. Be respectful, and do not attack the author, people mentioned in the article, or other commenters. Take on the idea, not the messenger.
  2. Don't use obscene, profane, or vulgar language.
  3. Stay on point. Comments that stray from the topic at hand may be deleted.
  4. Comments may be republished in print, online, or other forms of media, per our Terms of Use.

We are not able to monitor every comment that comes through (though some comments with links to multiple URLs are held for spam-check moderation by the system). If you see something objectionable, please let us know. Once a comment has been flagged, a staff member will investigate.

We accept clean XHTML in comments, but don't overdo it and please limit the number of links submitted in your comment. For more info, see the full Terms of Use.

Speak Your Mind

*