All targeted hacking starts with reconnaissance. Social engineers, red teams, and individual pentesters also collect information about their goals before moving on to action. Dozens of tools and hacks help them. Under the cut are links to some of them.
The post consists of 8 voluminous sections:
metasearch engines and search engines;
tools for working with dorks;
search by e-mail and logins;
search by phone numbers;
search in the TOR network;
search on the Internet of things, IP, domains and subdomains;
search for data on vulnerabilities and indicators of compromise;
source code search.
This list contains the tools that our team members use in their work. And yet, this selection will be useful not only for pentesters, but also for developers, journalists, HR, marketers and everyone who searches a lot on the Internet. Knowledge is power. Use them for good.
—
As in the case of tools for malware analysis or computer forensics , I have grouped the tools into groups for convenience, in accordance with my ideas of beauty. The list is certainly incomplete, and some links could well be in other sections. Since we are talking about search, the selection did not include numerous scanners and other tools for actively exploring computer networks, with the exception of those that can passively search for open sources.
Metasearch engines and search engines
Online Services
Hopain Tools , Inteltechniques , IntelligenceX , Aware OSINT Tools - are home pages for dozens of general and special purpose search engines. All of them are explicitly focused on OSINT.
Fagan Finder - a panel with dozens of search engines for libraries, archives and databases.
Dogpile , iZito , zapmeta are metasearch engines that aggregate results from Google, Yandex, Bing and other popular search engines and display them on one page.
Metaosint is a search engine for search engines. Provides a user-friendly interface for finding other tools.
Synapsint is a metasearch engine with the ability to search by IP, SSL, ASN, CVE, email and phone numbers.
ThatsThem is a combined name, address, phone, email and IP search engine.
Carrot 2 is a search engine with a built-in text clustering algorithm . Automatically combines thematically related sources into groups.
Isearchfrom - simulates search queries to Google from different countries. Helps to understand how regional restrictions affect search results.
Answerthepublic is an English-language search engine that returns common search queries for a given phrase.
Utilities
C - search from the command line in 106 sources.
SpiderFoot is a tool for automating search queries and exporting results to CSV, JSON, GEXF. Tailored for red team tasks. Equipped with a built-in web interface.
Query-Server is a tool to send search queries to Google, Yahoo, Bing, Ask, DuckDuckGo, Baidu, Exalead, Quora, Parsijoo, Dailymotion, Mojeek and Youtube and write the results to CSV, JSON or XML.
Recon-ng is an open source intelligence framework. Allows you to search for almost everything from logins, phone numbers and addresses, to files with financial statements that have fallen into the public domain. Outwardly, it resembles the Metasploit Framework.
Querytool is an OSINT tool based on google spreadsheets. Designed for advanced searches for people, email addresses, files, and more.
Maltego Community Edition is a free version of the OSINT tool for collecting information from Whois, search engines, social networks and identifying correlations between people, email addresses, logins, companies, websites, domains, etc.
sn0int is a framework for collecting and semi-automatically processing information about subdomains, IP addresses, compromised accounts, phone numbers, and social media profiles.
theHarvester is a console utility for collecting information for reddtiming. Allows you to perform active and passive reconnaissance using various search engines. Collects names, email addresses, IP addresses, subdomains and URLs.
ReconSpider is a crawler for finding IP addresses, emails, websites, organizations and searching for information from various sources.
Mr.Holmes is a tool for collecting information about user domains, names and phone numbers using public sources and Google Dorks.
Datasploit is an OSINT tool found on Kali or BlackArch Linux . Designed to collect data for a specific domain, email, username or phone number and save reports in text files, HTML and JSON.
YaCy is an open source decentralized search engine. Allows you to deploy your own search engine.
Tools for working with dorks
Online Services
Utilities
pagodo - automates the search for potentially vulnerable web pages using dorks from the aforementioned Google Hacking Database.
Grawler is a web-based PHP utility for automating the use of Google Dorks, cleaning and saving search results.
DorkScout is another dork search automation tool. Written in Golang.
oxDork is a utility for finding vulnerabilities and misconfigurations of web servers.
ATSCAN SCANNER - designed for searching using dorks and mass scanning web resources for vulnerabilities.
Fast Google Dorks Scan is an automated tool for collecting information about a specific website using dorks.
SiteDorks is a ready-made set of search queries for Google, Bing, Ecosia, DuckDuckGo, Yandex, Yahoo and so on. Includes 527 websites.
Search by email and logins
Online Services
Snusbase - indexes information from leaks and provides access to search for compromised email addresses, logins, names, IP addresses, phone numbers and password hashes.
have i been pwned? - search engine for data leaks. Allows you to check which incidents involved a particular email address.
Hunter and Skymem - search for corporate email addresses by URL.
whatsmyname - search for accounts in various services by login. The service is based on public JSON .
User Searcher is a free tool that will help you find a user by login on more than 2,000 websites.
CheckUserNames , Instant Username Search , Namecheckr , peekyou , usersearch — online services for searching user accounts by login.
Utilities
Infoga is a tool to collect email account information from public sources (search engines, PGP keyservers, Shodan) and check if email has been leaked using the haveibeenpwned.com API.
Holehe OSINT - checks if email is linked to accounts on sites like twitter, instagram, imgur. Supports over 100 portals. Uses the password recovery feature.
Mailcat - Looks up email addresses by nickname from 22 email providers.
WhatBreach is an OSINT tool that simplifies the task of finding leaks involving a specific email address. Able to load public databases.
h8mail and pwnedOrNot are tools for finding passwords from compromised email addresses in public databases.
Sherlock is a tool for searching social media accounts by username.
Snoop Project is a login search tool. According to the developer, it covers more than two and a half thousand sites.
Maigret - collects a dossier on a person login, checking accounts on two and a half thousand sites and collecting all available information from web pages. API keys are not required. Fork Sherlock.
Social Analyzer is an API, command line interface and web application for analyzing and searching human profiles on over 1,000 websites.
NExfil is a python utility for searching profiles by username on 350 websites.
SPY is another fast account name search engine that works with 210 websites.
Blackbird is a tool for searching accounts by login in social networks.
Marple - simplifies login search through public search engines from Google to Torch and Qwant.
GHunt is a modular tool for collecting data about google accounts.
UserFinder is a tool for finding profiles by username.
Search by phone numbers
Utilities
Moriarty is a utility for reverse (reverse) search by phone numbers. Allows you to find the owner, get links, social network pages and other information related to the number.
Phomber - searches for phone numbers on the Internet and extracts all available data.
PhoneInfoga is a well-known tool for finding international phone numbers. It first returns standard information such as country, region, carrier for any international telephone number, and then looks for traces of it in search engines to help identify the owner.
kovinevmv/getcontact is a utility for getting information from the databases of the GetContact application (not suitable for parsing, it allows you to perform only a limited number of requests).
Search in the TOR network
Online Services
Utilities
TorBot is a handy Onion crawler. Collects URLs and page titles with short descriptions, retrieves email addresses from sites, checks if links are active, and saves reports in JSON. Can be run in Docker.
VililantOnion - Onion crawler with keyword search support.
Katana-ds is a Python search automation tool using Google Dorks and with TOR support.
OnionSearch is a Python3 script for automating searches in the .onion zone through public services.
Devils Eye is an OSINT tool for searching the Darkweb. Does not require TOR installation. Can also search i2p network.
Search by IoT, IP, domains and subdomains
Online Services
Shodan is a famous search engine for collecting information about devices connected to the Internet.
Censys Search , GreyNoise , ZoomEye , Netlas , CriminalIp are IoT-focused search engines similar to Shodan.
Buckets by Grayhatwarfar is a public searchable database of open AWS Buckets, Azure Blobs, Digital Ocean Spaces.
Public buckets - Search for public AWS S3 & Azure Blob buckets.
macaddress.io , MAC Vendor Lookup , maclookup.app - determine the device manufacturer by MAC address, OUI or IAB.
CIRT , Default Password Lookup , Router Password , Open Sez Me - search through databases of passwords installed by default on various devices.
sitereport.netcraft - Provides a comprehensive summary of registration data and technologies used on a website.
IPVoid is a set of tools for exploring IP addresses: blacklist check, Whois, DNS lookup, ping.
who.is , DomainDossier , whois.domaintools - search by registration data and Whois.
DNSDumpster is a domain exploration tool that can discover hosts associated with a domain.
ip-neighbors - specifies the location of the server, and the names of hosts that share an IP address with it.
ShowMyIP - bulk search for IP addresses, allows you to check up to 100 IP addresses at the same time. The issue can be downloaded and saved to a .csv file.
MX Toolbox is another feature rich tool that allows you to search by domain name, IP address or hostname.
DNSViz is a collection of tools for analyzing and visualizing the Domain Name System. Has an open source code .
infosniper , ip2geolocation , ip2location , ipfingerprints , whoismind are search engines that allow you to find the approximate geographic location of an IP address, as well as other useful information, including ISP, time zone, area code, etc.
webmeup , openlinkprofiler , Meet Link Explorer - search for backlinks.
RapidDNS is a DNS query tool that makes it easy to find subdomains or sites with the same IP address.
CTSearch , crt — search for SSL/TLS certificates issued for a specific domain.
Utilities
IVRE is a network intelligence framework. An alternative to Shodan, ZoomEye, Censys and GreyNoise.
OWASP Amass is a network scanner with the function of searching for information in open sources. Aggregates information from dozens of different search engines and databases.
Infoooze is a NodeJs-based OSINT tool. Combines a port and subdomain scanner, DNS lookup URL scanner, Whois lookup and a number of other tools.
Automater is a utility for finding URLs, IP addresses, and MD5 hashes, designed to simplify the work of information security analysts. Installed on Kali Linux by default.
Raccoon is a tool designed for intelligence and information gathering with an emphasis on simplicity. Uses Nmap for port scanning and a range of passive data mining techniques to gather comprehensive information about the target.
Mitaka - designed to look up IP addresses, MD5, ASN and bitcoin addresses.
Photon is a scanner for working with information from open sources. Designed to scan specific websites for searches (pdf, png, xml, etc.), keys, subdomains and more. Exports the received data to JSON.
AttackSurfaceMapper is a scanner with open source search functionality. Looks for subdomains and their associated IP addresses.
HostHunter - Uses open source intelligence techniques to match IP addresses to hostnames. Exports search results to CSV or TXT files.
Subfinder is a modular tool for discovering subdomains using passive reconnaissance methods. Designed for bug hunters and pentesters.
Sublist3r - designed to search for subdomains using OSINT.
WASP Amass - performs network mapping using information from open sources.
Anubis is another utility for discovering subdomains and collecting information about them from open sources.
DOME is a python script that performs an active and/or passive scan to get subdomains and look for open ports.
Belati is a tool for collecting public data and documents from websites. Inspired by Foca and Datasploit .
Finding Vulnerability Data and Indicators of Compromise
Online Services
MITER CVE is a search engine, database and generally accepted vulnerability classifier.
NIST NVD - Search the official US government vulnerability database.
GitHub Advisory Database - A database of vulnerabilities that includes CVEs and security advisories.
CVEDetails , osv.dev , VulDB , maltiverse are a number of other sources of data on vulnerabilities and indicators of compromise.
opencve.io is a CVE search engine with built-in new threat alerts.
security.snyk.io and Mend Vulnerability Database , Vulncode-DB are open source vulnerability databases.
Cloudvulndb is a project that accumulates vulnerabilities and security issues of cloud service providers.
Vulnerability Database is a system for finding information about current threats.
Rapid7 - DB is a database that contains details of more than 180,000 vulnerabilities and 4,000 exploits. All exploits are included in Metasploit.
Exploit DB is a CVE-compliant archive of publicly available exploits and vulnerable software.
sploitus is a search engine for exploits and hacking tools.
Search by source code
Online Services
Searchcode - Search for real examples of functions, APIs, and libraries in 243 languages on GitHub, GitLab, Bitbucket, GoogleCode, and other source code repositories.
Sourcegraph is a fast and functional search engine for open-source repositories. Has a selfhosted version of .
HotExamples - search for code examples of an open-source project. Allows you to see on one page examples of the use of a particular class or method from several projects.
Libraries.io - Search 4,690,628 packages across 32 package managers.
RepoSearch - search for source code in SVN and GitHub repositories.
grep.app - search through Git content. Useful for finding strings related to vulnerabilities, indicators of compromise, and malware.
Search from Cyber_detective - Search 20 open source repositories using dorks.
PublicWWW - allows you to find any alphanumeric fragment, including pieces of HTML, JavaScript, CSS code in the code of web pages.
NerdyData - Search for websites using certain technologies.
Utilities
Gitrob is a tool that helps you find potentially sensitive files placed in public Github repositories. Clones the repository, iterates through the commit history, flags suspicious files, and outputs the results to the web interface.
Github Dorks is a utility for searching using dorks via the GitHub Search API. Written in Python.
gitGraber is a tool for monitoring GitHub and quickly searching for sensitive data, such as credits from Google, Amazon (AWS), Paypal, Github, Mailgun, Facebook, Twitter, Heroku, Stripe, Twilio.
github-search is a collection of console tools for Github research.
TheScrapper is designed to search for email addresses and social media accounts in the source code of a website.
