Collecting Open-Source Intelligence

What Is This

Collecting Open-Source Intelligence (OSINT) is the process of gathering, analyzing, and synthesizing publicly available information from a wide range of sources to build actionable intelligence on threat actors, their infrastructure, and ongoing attack campaigns. This skill enables cybersecurity professionals to leverage open data-including domain registrations, certificate transparency logs, public threat feeds, paste sites, and dark web forums-to identify, track, and contextualize adversarial activity without direct interaction with target systems. The collecting-open-source-intelligence skill utilizes both passive reconnaissance tools and dark web monitoring techniques to support a variety of threat intelligence, red teaming, and incident response use cases.

The skill is designed to activate for tasks involving tools and frameworks such as Maltego, Shodan, Recon-ng, SpiderFoot, and the OSINT Framework. It is aligned with cybersecurity best practices and standards, including the MITRE ATT&CK technique T1591 (Gather Victim Network Information), and relevant controls from the NIST Cybersecurity Framework (CSF) such as ID.RA-01 (Asset Identification), ID.RA-05 (Threat and Vulnerability Identification), DE.CM-01 (Detection Processes), and DE.AE-02 (Detection Activities).

Why Use It

Threat actors are constantly evolving their infrastructure and attack tactics. Relying solely on internal logs or proprietary threat feeds can leave organizations blind to emerging risks and external exposures. By systematically collecting OSINT, defenders can:

• Build a comprehensive threat landscape map by identifying external infrastructure associated with adversaries.
• Enrich Cyber Threat Intelligence (CTI) reports with real-time, publicly observable indicators such as IP addresses, domains, SSL certificates, or ASN ownership.
• Support incident response efforts by correlating attack infrastructure with known threat actor tactics, techniques, and procedures (TTPs).
• Conduct non-intrusive reconnaissance for red team assessments, helping organizations understand their digital footprint and potential attack surfaces.
• Monitor dark web and underground forums for mentions of organizational assets or credentials, providing early warning of targeted campaigns.

The collecting-open-source-intelligence skill ensures that intelligence gathering remains passive by default, reducing legal and ethical risks while maximizing information gain from open sources.

How to Use It

To effectively collect OSINT, follow these core steps:

1. Define Collection Objectives
   Determine the intelligence requirements-such as profiling a phishing campaign, mapping adversary-owned infrastructure, or discovering leaked credentials.

2. Select Appropriate Tools
   The skill supports a variety of tools, including:

   • Maltego: Graphical link analysis tool for mapping relationships between entities (domains, IPs, email addresses).
   • Shodan: Search engine for internet-connected devices, useful for identifying exposed services and vulnerable systems.
   • Recon-ng: Modular web reconnaissance framework, ideal for automating data collection.
   • SpiderFoot: Automated OSINT collection and analysis platform.
   • OSINT Framework: Directory of tools and resources for targeted intelligence gathering.

3. Passive Data Collection
   Gather information using methods that do not directly interact with the target system. Example code for passive DNS lookup with Python:

   python

   import requests
   
   domain = "example.com"
   url = f"https://api.hackertarget.com/dnslookup/?q={domain}"
   response = requests.get(url)
   print(response.text)

   This script queries HackerTarget's API for DNS records associated with a domain, supporting passive mapping of infrastructure.

4. Analyze and Synthesize Data
   Correlate data across multiple sources to identify patterns, relationships, or anomalies. For example, combining WHOIS data with SSL certificate transparency logs can reveal additional domains registered by the same threat actor.

5. Document Findings
   Clearly record the intelligence collected, including indicators of compromise (IOCs), infrastructure maps, and assessed threat actor TTPs. Use structured formats like STIX/TAXII where possible.

When to Use It

Activating this skill is recommended in the following scenarios:

• Threat Actor Profiling: When you need to build or update profiles on adversaries using public indicators and behavioral evidence.
• Infrastructure Reconnaissance: To map command and control (C2) servers, phishing domains, or malware distribution sites linked to a campaign.
• Red Team Engagements: During authorized and scoped assessments to understand the organization's external attack surface.
• Incident Response: While investigating a breach or suspicious activity, use OSINT to pivot from internal logs to external infrastructure or actor-linked assets.
• Threat Enrichment: To supplement internal alerts or external threat feeds with additional context and indicators from open sources.

Important Notes

• Authorization and Scope: Only collect OSINT within the boundaries of legal, ethical, and organizational guidelines. Passive reconnaissance means no direct probing or scanning of target systems unless explicitly authorized.
• Data Validation: Publicly available data can be incomplete, outdated, or intentionally misleading. Always corroborate findings with multiple sources.
• Operational Security (OpSec): When investigating criminal or underground forums, use anonymization techniques (VPNs, Tor, burner accounts) to prevent attribution and safeguard analyst identity.
• Compliance: Adhere to relevant standards such as NIST CSF and ensure that data handling meets organizational and regulatory requirements.
• Skill Limitations: This skill focuses on passive collection and synthesis. For active engagement or deeper penetration testing, use additional skills and obtain proper authorization.

By leveraging the collecting-open-source-intelligence skill, cybersecurity professionals can enhance their threat intelligence capabilities, improve situational awareness, and proactively defend against evolving external threats.