Open Source Intelligence

Open source intelligence, sometimes abbreviated OSINT, is basically a method of gathering information from various publicly available (and thus open source) places, as opposed to covert or classified sources, or data gathered and purchased from another source. Open sources for intelligence include everything from the media to public government data, academic papers and maps, Internet forums and social networking--pretty much anything that's publicly available.

It does not refer to the use of open source software, though there are some uses for open source software within open source intelligence gathering and analyzing. Nor does it refer to the intelligence gathered and analyzed being at all open to the public, as often it lies within the proprietary or government realm.

Before the Internet became an everyday thing, open source intelligence was pretty much non-existent. There were no real bulk repositories of public information, let alone ones that could be searched instantly and effectively. Even powerful information such as that contained within national censuses were difficult to go through.

Additionally, the sort of data required for many fields of research, such as the everyday interactions of everyday people, just wasn't compiled in any one place. Analysts had to pour through public paper documents, often having to physically travel to different information centers to get the information they needed.

It was still possible to get relevant, powerful information using open source intelligence techniques, however. A great example occurs way back in WWII, when the US military would look at the prices of various products in cities to see whether they had successfully destroyed lines of transportation—such as checking out the price of oranges in Paris to see if a certain bridge had been taken out. However, the possibilities were limited.

The Internet changed all that. More information than could ever possibly be analyzed is uploaded on a daily basis, from public Twitter statuses to public blogs. While individually this information may be useless, in bulk it has incredible potential. For instance, a recent earthquake made the natural disaster reporting potential of Twitter a reality.

While some types of information are still out of the open source intelligence domain, the gap is closing fast as businesses and governments trend towards more transparent operations, where their every doing is made public.

The US government was one of the first to really start using open intelligence techniques, and they continue to do so. From tracking potential suspects without violating their (official) privacy to keeping closer tabs on the public attitude on politics, the government is increasingly online and doing research. While individual departments tend to have their own data gathering specialists, the Federal Research Division specializes in open source research services for governmental purposes. All the same, the government has also been criticized for not using open source intelligence enough—especially in one of its more critical applications, national security.

The US government is also making an effort to become more transparent—a term often referred to as open government. Budgets, proposals, amendments, everything is being placed online for everyone to see. This only increases the pool of information available for open source intelligence gathering.

Journalists have been using open source intelligence techniques for as long as there have been newspapers, combing through whatever information they can get their hands on to glean insights into the doings of the world. With the Internet, both comparing their findings to other journalists—and contrasting often opposing views—makes for a more dynamic, more competitive dialogue.

There are numerous businesses worldwide that specialize in open source intelligence, from gathering the data in the first place to analyzing it in detail. They are often used by other businesses who wish to outsource this process to another company, for purposes of creating better products for their customers by everything from analyzing feedback on their Facebook statuses to checking out forums.

Academia, too, has uses for open source intelligence. Open science has tremendous potential to accelerate research to previously unknown levels as everything from academic journals to basic chemistry lab results is put on the Internet and made public for everyone to see—and analyze. Even information gleaned through social networking has a place here, especially in the field of psychology where personalities and how they express themselves may be analyzed through an array of techniques. Another example, NASA routinely posts its latest astronomy pictures online with both the intent of sparking public interest and scientific inquiry.

At current, it's a safe bet that many of the products and pieces of news that you see around you on a daily basis are products of open source intelligence. Clearly, this is a concept that is at home in the digital age, and one that will doubtless prove infinitely useful as mass data techniques become more sophisticated.

For more information on open source intelligence, check out this article.