Social Media and Open Source Intelligence

Image Source: http://www.expertsystem.com/wp-content/uploads/2016/06/osint-big-data.jpg 1

Value of OSINT

Information does not have to be secret to be valuable.  Open

Source Information is unclassified, available and one of the
largest sources of information available

In many cases not leveraged to it's full potential due to a number

of issues including volume, language, format and insight (the
ability to understand what has been collected)

2
Recent History of OSINT
The Foreign Broadcast Information Service (FBIS) was created in 1941 to
access and exploit OSINT in relation to World War II. A classic example
of their value and success is reflected in the price of oranges in Paris as
an indicator of whether railroad bridges had been bombed successfully.

The recent history of OSINT began in 1988 when General Alfred M. Gray,
Jr., Commandant of the Marine Corps, called for a redirection of US
intelligence away from the collapsing Soviet Union and toward non-state
actors and Third World zones of instability. Additionally, he pointed out
that most of the intelligence which needs to be known could be obtained
via OSINT, and recommended a substantive increase in resources for this
aspect of the intelligence collection spectrum of sources.

Source: https://en.m.wikipedia.org/wiki/Open-source_intelligence 3
 Recent Perspective on OSINT
“Classified sources and methods will always have value in our
agency and to our customers, but we cannot always view
unclassified information as supplemental”

“Moving forward the reverse is more likely to be true - that

which is exquisite but classified will supplement an ever
broader and richer unclassified base”

NGA Director Robert Cardillo

Source: CSIRNET.com “Open Source Intel” by Adam Stone, May 2016

4
Contents
• OSINT vs SOCMINT
• Aggregation vs Live Connection
• Deep and Dark Web
• Peer to Peer Networks
• Common Issues
• Types of Analysis
• Practical Applications

5
OSINT vs SOCMINT
Open Source Intelligence (OSINT) – Information gathered from
unrestricted or public sources. Typically not focused on Social Media
Information (i.e. person or group centric) though often encompasses
information feeds that are resident within some social media
platforms.

Social Media Intelligence (SOCMINT) – Information gathered from

Social Media Platforms. It is person and group centric and focuses on
networks, messaging, and social network analysis.

6
Social Media Volumes
Tweets - 500 Million / Day

Active Users / Monthly*

Facebook (1.79 billion) - Avg ~155 friends/user**

WhatsApp (500 million)

Twitter (284 million)

Instagram (200 million)

*Source: https://zephoria.com/top-15-valuable-facebook-statistics/
** Source: http://expandedramblings.com/index.php/by-the-numbers-17-amazing-facebook-stats/
7
Social Media
While not “vetted”, can be significantly faster, with greater volume and
more granular than other forms of intelligence

8
Aggregation vs Live Connection
• Aggregation Services focus on
content, metadata, and trends
analysis.
• They analyse historical information
and are helpful when searching by
content and sentiment.
• They offer limited utility for granular
network analysis as the entire network
is not captured at the time content is.
• Example – GNIP, Voyager, Babel
Street, Pathar.
• Live Connection Services focus on
Individuals, Groups, and Networks.
• They analyse networks and content as they
occur at the time of capture and are helpful
when searching by named individual or
individuals associated with a group.
• The application of Social Network Analytics
yields insight into the position and roles of
individuals.
• Example – X1, SNAPD, API Direct.
9
The Open Source Terrain
It’s not just Social Media

• News
• Blogs
• Marketplaces
• Wikis
• Business
• Reviews
• Events
…

Image Source: https://conversationprism.com/ 10

OSINT News Media Example: GDELT
(Global Database of Events, Language and Tone)

GDELT monitors print, broadcast, and web news media in over 100
languages from across every country in the world to keep continually
updated on breaking developments anywhere on the planet. Its
historical archives stretch back to January 1, 1979 and update every 15
minutes. Through its ability to leverage the world's collective news
media, GDELT moves beyond the focus of the Western media towards a
far more global perspective on what's happening and how the world is
feeling about it.

11
 Surface Web

Deep Web Dark Web

Unindexed by Technological
Search Engines barrier to entry,
Shrouded behind requiring
password/paywall software for
protection access

12
Comparing the Surface Web - Deep/Dark Web
• Crowded social commentary
• News reports
• Opinion pieces
• Mainstream perspectives
• Dialogue between threat actors
• Black market products and services
• Malicious tactics, techniques, and
procedures (TTPs)
• Weapons and training manuals
• Illicit community perspectives
13
What’s on the Deep & Dark Web?
• Drugs
• Counterfeit Items
• Stolen Goods
• Weaponry
• Identities
• Credit Cards
• Malware / Source
Code

14
Early Identification of Radicalization

15
Early Identification of Radicalization

16
Leveraging Deep / Dark Web

It is possible to generate Actionable

Intelligence through Deep and Dark Web
monitoring that allows your organization to
proactively protect itself and not rely solely on
signatures, behaviors, or indicators from other
breaches.

17
Analysis of Peer to Peer networks
for Terror Materials
Protocol! Client+examples!

Ares+ Ares%Galaxy,%Warez%P2P%

Bi3orrent+ Azureus,%BitComet,%BitTornado,%Transmission,%Vuze,%µTorrent%

Direct+Connect+ StrongDC,%DC++,%%

Open+Fas3rack+ OpenFT,%GIFT,%KcEasy,%KGNitro%

eDonkey+ eMule,%AMule%

Gnutella+ BearShare,%GTKGgnutella,%LimeWire,%%Shareaza,%Phex,%Frostwire%

Gnutella+2+ Shareaza,%Gnucleus,%%
18
Peer to Peer Network Functionality

19
Discovery / Tracking of Terrorist
Materials on Peer to Peer Networks

20
Examination of the individual
materials

21
Common Issues with OSINT
• Volume: Too much to read

• Language: Not just English

• Format: Largely Unstructured in many cases

• Insight: Understanding what we’ve collected

22
Volume
Since it’s often
impossible to “save
everything”, one
approach to addressing
this is “streaming
analaytics”, providing the
ability to run analytics
across the data as it
moves to determine if
this information is
valuable

23
Language
A large amount of
content we’re collecting
will not be in english
therefore we need to
translate that content
to make it easier to
understand

24
Format
Frequently the
content we collect
will be unstructured,
so we need a means
to “understand”
what’s been written
by creating a “mental
model” of the
information by
extracting entities,
links and properties

25
Insight

Once we have collected and processed all this

information, what does it all mean?
To understand what we’ve collected and processed we
may require multiple forms of analysis to gain insight into
what we have collected and what it means to us.

26
Link Analysis
Evaluating relationships (connections) between nodes

27
Social Network Analysis
• Relationships: Obvious, Non
Obvious
• Anchor Points: Home, Work,
Religion, relationships, support
infrastructure
• Behavior: Rational, Irrational,
Open, Closed norms and
variation
• Constraints: Political, cultural,
physical
• Communication: Social networks
to understand the conscious/
cognitive dimension of Human
Geography
• Movement: Travel Home, Work,
Relationships, as a human
activity
• Culture: Norms / variation like
language, religion, economy,
gov’t
• Challenges: Alliances & relations
including close, casual,
competitive, adversarial
28
Relationship Analysis
Fusing and
Correlating
information to
identify
relationships that
have not been
disclosed

29
Identity Analysis
Using the
“disclosed”
information to
identify alternate
identities that may
be in use

30
Geo-Spatial Analysis
Understanding
information in a
geo-spatial context
in relation to other
known information

31
Temporal Analysis
Understanding
how events
relate over time

32
Frequency/Statistical Analysis

33
OSINT Law Enforcement Applications
Early Identification of Self Radicalization - Identifying individuals as they begin the path to
radicalization through the analysis of the radicalization materials

Tax evader detection - Web content such as news, blogs and social networks, especially data
concerning famous companies and celebrities are of great value. Using OSINT applications
customized for the analyst’s requirements, it’s possible to retrieve precise and contextual
information from big data on the events, behaviors, lifestyles, activities, and professional and
personal relationships of a single target.

Online counterfeit (i.e. pharmaceuticals, clothing, jewelry). Text mining and OSINT applications
can extract the key elements present within a posting such as: vendor alias, email, telephone
number, brand, product, etc. This data is then compared to identify the typical characteristics
of a counterfeit product (i.e. available only in certain sizes) and to highlight the correlation
between the data (i.e. a connection between vendors and/or repeated events between
vendors).

Content Source: http://www.expertsystem.com/osint-applications-3-examples/ 34

OSINT Military Applications
• TargetAnalysis: discusses how OSINT might fulfill team needs in the absence of classified intelligence support, to
create a detailed description and vulnerability assessment, evaluate the natural environment and the human
environment, and carry out route planning.

• Terrain
Analysis: uses OSINT to establish key factors relevant to special aviation and covert ground movement, in
part by leveraging commercial charts, commercial imagery, and alternatives for terrain reconnaissance including
unmanned aerial vehicles and indigenous scouts.

• CivilAffairs: can use OSINT in relation to human intelligence (understanding the demographics, the socio-
economic environment, displaced persons, and crime, among other topics); to technical intelligence about the
local command & control, communications, computing, and intelligence environment, the infrastructures of
transportation, power, and finance; to welfare intelligence (water, food, medical); cultural intelligence about
protected or restricted targets, and liaison intelligence.

• WeatherAnalysis: uses OSINT as a means of rapidly getting to the basics of temperature, visibility and timing of
sun and moon, wind, and inclement weather.

Source: http://www.oss.net/dynamaster/file_archive/060409/5432a5e19def62b82684a111fe03f899/STEELE%20OSINT%20FOR%20HANDBOOK%203.3%20Chapter.doc
35
OSINT Civil Applications
Visa Processing: Evaluating OSINT and Social Media sources to determine
if a visa application requires additional scrutiny

Employee Vetting: Leveraging OSINT to evaluate the veracity of the

information you have been provided

Know your customer: Financial Responsibility to understand who you are

doing business with

36
Thank You

37
OSINT References
https://inteltechniques.com/links

https://www.toddington.com/resources/

http://www.onstrat.com/osint/

http://i-sight.com/resources/101-osint-resources-for-
investigators/

https://www.cia.gov/library/center-for-the-study-of-
intelligence

38