The term OSINT is defined as the collection, processing, analysis, production, classification, and dissemination of information whose sources are publicly available, lawfully obtained and employed by the public in response to official national security requirements. This article addresses the origins of OSINT as an intelligence discipline, arguing that it should be better known as an espionage technology, and its contribution to an integrated, all-source knowledge management process within the intelligence enterprise.
History of OSINT
The history of exploiting publicly available information dates back to the emergence of intelligence as a tool to support government decision-making and operations. However, this was not a systematic effort until the United States took the lead in institutionalizing and professionalizing the ability to independently monitor foreign media, creating the Foreign Broadcasting Monitoring Service (FBMS), a research initiative by Princeton University. produced. After the Japanese attack on Pearl Harbor, the FBMS quickly gained momentum. In 1947, it was renamed the Foreign Broadcasting Intelligence Service (FBIS) and was attached to the newly created Central Intelligence Agency (CIA). In 2005, following the 9/11 terrorist attacks and the passage of the Intelligence Reform and Terrorism Prevention Act, the Federal Bureau of Investigation (FBIS), along with other research elements, was transformed into the Director of the National Intelligence Open Resource Center (OSC). Since its establishment, the Information Service has been responsible for filtering, transcribing, translating and archiving news items and information from various foreign media.
In 1939, the British government asked the BBC to launch a civilian, and later commercial, service to censor foreign print news and broadcasts through its Digest of Foreign Broadcasts, which later It was named “Summary of World Broadcasts” (SWB), which is now BBC Monitoring. Their aim, as stated in a 1940 BBC handbook, was to build a “modern Tower of Babel, where, with exemplary focus, they listened to friend and foe.” By 1943 In mid-year, the BBC monitors 1.25 million broadcast words a day. From 1947 to 1948, the British Broadcasting Corporation (BBC) established a formal partnership with its American counterpart, and the two sides agreed to a full exchange of output content. Also in 1948, the US Library of Congress established a research unit based on the Aeronautical Research Unit, using the library’s vast collections to provide custom research and analysis services. It is now called the Federal Research Department (FRD).
During the Cold War, countries on both sides of the Iron Curtain built open-source collection capabilities, often embedded in their covert intelligence agencies. According to CIA analyst Stephen Mercado, public sources of information not only “form a major part of all intelligence,” but ultimately become “the primary source of information” about adversary military capabilities and political intentions, including early warning and Threat prediction. For example, the East German Ministry of State Security (MfS) analyzes 1,000 Western magazines and 100 books each month, while also summarizing more than 100 newspapers and 12 hours of West German radio and television broadcasts.
During the Cold War, open source was already a well-established source of information, often the means of choice for other collections, or, as Joseph Nye put it, “the outer pieces of the jigsaw puzzle”. With the development of Internet technology, public information has had a huge impact on all aspects of modern political, social and economic life. However, we need to realize that the Internet is not itself a source (except for its metadata); rather, it is a means of transmitting information and virtual locations.
Most intelligence agencies have been slow to recognize the value of the Internet for two reasons:
(1) Intelligence agencies seek information advantage by secretly processing intelligence. Relying on open information and its corresponding copyright restrictions runs counter to this idea.
(2) In most cases, it is more difficult, risky and expensive to employ covert methods in order to obtain secret sources, thereby giving the impression that these sources must be of higher value than open sources, confusing methods and products, or mistake confidentiality for knowledge.
After the collapse of the Soviet Union, Western intelligence agencies shifted their operations to new geographic and thematic focuses, such as Africa and Asia, non-state actors, low-intensity conflict in expeditionary settings, political and religious terrorism, weapons of mass destruction (WMD) ) and the fragility of computer networks, which has led to a greater emphasis on open source. The U.S. military first coined the term OSINT in the late 1980s, arguing that intelligence reform was necessary to respond to dynamic information needs, especially at the tactical level on the battlefield. In 1992, the Intelligence Reorganization Act defined the goal of information gathering as “to provide timely, objective, and unbiased intelligence based on all sources available to U.S. intelligence agencies, whether public or private.” In 1994, The Community Open Source Projects Office (COSPO) was established within the CIA. In 1996, the Committee on the Roles and Capabilities of U.S. Intelligence Agencies (often referred to as the Aspin-Brown Committee) concluded: “We should also make greater efforts to take advantage of the vast amount of information now available from open sources.” Parallel efforts to develop a framework using OSINT have resulted in the publication of several manuals, introductory and practical manuals of varying quality. Using the European Media Monitor (EMM) and the OSINT suite, among other tools and projects, the European Union (EU) Commission Joint Research Centre (JRC) is developing its own tools to meet the challenges posed by the growing flood of information.
9/11 proved to be a watershed moment for OSINT, with the 2004 National Committee on Terrorist Attacks in the United States (9/11 Committee) recommending the creation of an open source agency, but no further comment or details. The concept was adopted in 2005 — along with related recommendations from the U.S. Weapons of Mass Destruction Intelligence Capability Council (WMD Committee) — when the Director of National Intelligence (DNI) established the OSC to bring together the CIA’s federal investigations into Bureau (FBIS) merged with World News Connection (WNC) under the supervision of the National Technical Information Service (NTIS). OSC describes itself as “the U.S. government’s premier provider of open intelligence, [and] providing information on foreign political, military, economic and technological issues beyond conventional media from an expanding world of open sources.” , and also appointed the Assistant Deputy Director of the National Open Source Intelligence (ADDNI/OS), increasing the visibility of the national open source enterprise. With the development of regional fusion centers focused on homeland security and law enforcement issues, OSINT is the primary source for consolidating relevant intelligence into actionable products.
OSINT and the private sector
From an economic point of view, national security, as a public good, can only be effectively provided by the government or under state oversight. Despite its great value, OSINT does not require special permissions. Because non-state contractors may be superior in the ability and resources to deliver OSINT, they can contribute to a better delivery of national security. Intelligence obtained from open but unlawful sources or the use of means should not be considered classified intelligence, such as the disclosure of classified or proprietary information of questionable legal status. The CIA’s venture capital firm In-Q-Tel’s investment in Recorded Future, a predictive analytics tool for network surveillance, confirms what former CIA Director Michael Hayden said: “Confidential information doesn’t always matter in our industry.”
For OSINT, a key issue for government-private sector collaboration is the need for secrecy regulations to protect national security. Sometimes intelligence products based solely on publicly available information must be kept secret to protect government interests from disclosure. Intelligence agencies must integrate and control outreach and contractor efforts to prevent compromising operations and national security. Collaboration with academia can avoid potential conflicts between the state and profit-oriented players. Universities are fertile ground for acquiring expertise in the public domain and can be ideal partners for intelligence agencies.
The fact that open sources tend to provide most of the intelligence input makes OSINT an important part of all-source intelligence efforts. Every intelligence professional should understand the sources and methods of OSINT, especially as analysis and collection increasingly converge. However, outreach activities and open source development must be supported by specialized elements to ensure analysts keep pace with emerging technologies and markets. OSINT’s specialist experts are best qualified to identify potential capability gaps and assess where contractors can use them. For example, a great way to incorporate private sector knowledge and skills into intelligence systems is the OSINT certification program, currently being pursued in the United States.
Challenges for OSINT
Due to its openness, OSINT can facilitate sharing. However, the means of sharing OSINT and more restricted intelligence categories need to be improved. This need exists not only in the national security sector, but also in those responsible for domestic security and law enforcement. Therefore, a vertically and horizontally consistent shared guarantee system must be established.
It is important to be public about the government’s credibility and to justify its decisions to the public and international allies. However, there is an inherent weakness if the enemy uses overt resources to undermine national security. OSINT can be used to assess a country’s vulnerability.
Hostile countries also manipulate open resources for deceptive purposes. However, in today’s world, where a lot of information is made public, such deception has become more difficult.
While the rapid development of information technology is an important challenge, the human element should not be underestimated. Ultimately, intelligence espionage techniques are always determined by human expertise. Collectors and analysts therefore need legal and practical training, appropriate literacy skills and first-class technical skills (such as data mining, web analysis and translation solutions) in order to put raw OSINT data into different contexts and understand they. With the emergence of new media on the Internet, the variety, quantity and speed of information have multiplied. The challenge today is no longer to “connect the dots,” but to organize the flow of information, distinguish signal from noise, and identify sources of information in a timely manner to support government policymakers and warfighters.
