github.com/stamparm/maltrail @2.1 sqlite

Content

• Introduction
• Architecture
• Demo pages
• Requirements
• Quick start
• Administrator's guide
• Sensor
• Server
• User's guide
• Reporting interface
• Real-life cases
• Mass scans
• Anonymous attackers
• Service attackers
• Malware
• Suspicious domain lookups
• Suspicious ipinfo requests
• Suspicious direct file downloads
• Suspicious HTTP requests
• Port scanning
• DNS resource exhaustion
• Data leakage
• False positives
• Best practice(s)
• License
• Sponsors
• Developers
• Presentations
• Publications
• Blacklist
• Thank you
• Third-party integrations

Introduction

Maltrail is a malicious traffic detection system, utilizing publicly available (black)lists containing malicious and/or generally suspicious trails, along with static trails compiled from various AV reports and custom user defined lists, where a trail can be anything from a domain name (e.g. zvpprsensinaix.com for Banjori malware), URL (e.g. hXXp://109.162.38.120/harsh02.exe for known malicious executable), IP address (e.g. 185.130.5.231 for known attacker) or HTTP User-Agent header value (e.g. sqlmap for automatic SQL injection and database takeover tool). Also, it uses (optional) advanced heuristic mechanisms that can help in the discovery of unknown threats (e.g. new malware).

The following (black)lists (i.e. feeds) are being utilized:

360bigviktor, 360chinad, 360conficker, 360cryptolocker, 360gameover, 
360locky, 360necurs, 360suppobox, 360tofsee, 360virut, abuseipdb, alienvault, 
atmos, badips, bitcoinnodes, blackbook, blocklist, botscout, 
bruteforceblocker, ciarmy, cobaltstrike, cruzit, cybercrimetracker, 
dataplane, dshieldip, emergingthreatsbot, emergingthreatscip, 
emergingthreatsdns, feodotrackerip, gpfcomics, greensnow, ipnoise,
kriskinteldns, kriskintelip, malc0de, malwaredomainlistdns, malwaredomains,
maxmind, minerchk, myip, openphish, palevotracker, policeman, pony,
proxylists, proxyrss, proxyspy, ransomwaretrackerdns, ransomwaretrackerip, 
ransomwaretrackerurl, riproxies, rutgers, sblam, socksproxy, sslbl, 
sslproxies, talosintelligence, torproject, trickbot, turris, urlhaus, 
viriback, vxvault, zeustrackermonitor, zeustrackerurl, etc.

As for static entries, the trails for the following malicious entities (e.g. malware C&Cs or sinkholes) have been manually included (from various AV reports and personal research):

1ms0rry, 404, 9002, aboc, absent, ab, acbackdoor, acridrain, activeagent, 
adrozek, advisorbot, adwind, adylkuzz, adzok, afrodita, agaadex, agenttesla, 
aldibot, alina, allakore, almalocker, almashreq, alpha, alureon, amadey, 
amavaldo, amend_miner, ammyyrat, android_acecard, android_actionspy, 
android_adrd, android_ahmythrat, android_alienspy, android_andichap, 
android_androrat, android_anubis, android_arspam, android_asacub, 
android_backflash, android_bankbot, android_bankun, android_basbanke, 
android_basebridge, android_besyria, android_blackrock, android_boxer, 
android_buhsam, android_busygasper, android_calibar, android_callerspy, 
android_camscanner, android_cerberus, android_chuli, android_circle, 
android_claco, android_clickfraud, android_cometbot, android_cookiethief, 
android_coolreaper, android_copycat, android_counterclank, android_cyberwurx, 
android_darkshades, android_dendoroid, android_dougalek, android_droidjack, 
android_droidkungfu, android_enesoluty, android_eventbot, android_ewalls, 
android_ewind, android_exodus, android_exprespam, android_fakeapp, 
android_fakebanco, android_fakedown, android_fakeinst, android_fakelog, 
android_fakemart, android_fakemrat, android_fakeneflic, android_fakesecsuit, 
android_fanta, android_feabme, android_flexispy, android_fobus, 
android_fraudbot, android_friend, android_frogonal, android_funkybot, 
android_gabas, android_geinimi, android_generic, android_geost, 
android_ghostpush, android_ginmaster, android_ginp, android_gmaster, 
android_gnews, android_godwon, android_golddream, android_goldencup, 
android_golfspy, android_gonesixty, android_goontact, android_gplayed, 
android_gustuff, android_gypte, android_henbox, android_hiddad, 
android_hydra, android_ibanking, android_joker, android_jsmshider, 
android_kbuster, android_kemoge, android_ligarat, android_lockdroid, 
android_lotoor, android_lovetrap, android_malbus, android_mandrake, 
android_maxit, android_mobok, android_mobstspy, android_monokle, 
android_notcompatible, android_oneclickfraud, android_opfake, 
android_ozotshielder, android_parcel, android_phonespy, android_pikspam, 
android_pjapps, android_qdplugin, android_raddex, android_ransomware, 
android_redalert, android_regon, android_remotecode, android_repane, 
android_riltok, android_roamingmantis, android_roidsec, android_rotexy, 
android_samsapo, android_sandrorat, android_selfmite, android_shadowvoice, 
android_shopper, android_simbad, android_simplocker, android_skullkey, 
android_sndapps, android_spynote, android_spytekcell, android_stels, 
android_svpeng, android_swanalitics, android_teelog, android_telerat, 
android_tetus, android_thiefbot, android_tonclank, android_torec, 
android_triada, android_uracto, android_usbcleaver, android_viceleaker, 
android_vmvol, android_walkinwat, android_windseeker, android_wirex, 
android_wolfrat, android_xavirad, android_xbot007, android_xerxes, 
android_xhelper, android_xploitspy, android_z3core, android_zertsecurity, 
android_ztorg, andromeda, antefrigus, antibot, anubis, anuna, apocalypse, 
apt_12, apt_17, apt_18, apt_23, apt_27, apt_30, apt_33, apt_37, apt_38, 
apt_aridviper, apt_babar, apt_bahamut, etc.

Architecture

Maltrail is based on the Traffic -> Sensor <-> Server <-> Client architecture. Sensor(s) is a standalone component running on the monitoring node (e.g. Linux platform connected passively to the SPAN/mirroring port or transparently inline on a Linux bridge) or at the standalone machine (e.g. Honeypot) where it "monitors" the passing Traffic for blacklisted items/trails (i.e. domain names, URLs and/or IPs). In case of a positive match, it sends the event details to the (central) Server where they are being stored inside the appropriate logging directory (i.e. LOG_DIR described in the Configuration section). If Sensor is being run on the same machine as Server (default configuration), logs are stored directly into the local logging directory. Otherwise, they are being sent via UDP messages to the remote server (i.e. LOG_SERVER described in the Configuration section).

Server's primary role is to store the event details and provide back-end support for the reporting web application. In default configuration, server and sensor will run on the same machine. So, to prevent potential disruptions in sensor activities, the front-end reporting part is based on the "Fat client" architecture (i.e. all data post-processing is being done inside the client's web browser instance). Events (i.e. log entries) for the chosen (24h) period are transferred to the Client, where the reporting web application is solely responsible for the presentation part. Data is sent toward the client in compressed chunks, where they are processed sequentially. The final report is created in a highly condensed form, practically allowing presentation of virtually unlimited number of events.

Note: Server component can be skipped altogether, and just use the standalone Sensor. In such case, all events would be stored in the local logging directory, while the log entries could be examined either manually or by some CSV reading application.

Demo pages

Fully functional demo pages with collected real-life threats can be found here.

Requirements

To run Maltrail properly, Python 2.6, 2.7 or 3.x is required on *nix/BSD system, together with installed pcapy-ng package.

NOTE: Please use pcapy-ng. The older pcapy library is deprecated and causes issues in Python 3 environments. Examples.

• Sensor component requires at least 1GB of RAM to run in single-process mode or more if run in multiprocessing mode, depending on the value used for option CAPTURE_BUFFER. Additionally, Sensor component (in the general case) requires administrative/root privileges.

• Server component does not have any special requirements.

Quick start

The following set of commands should get your Maltrail Sensor up and running (out of the box with default settings and monitoring interface "any"):

• For Ubuntu/Debian

sudo apt-get install git python3 python3-dev python3-pip python-is-python3 libpcap-dev build-essential procps schedtool
sudo pip3 install pcapy-ng
git clone --depth 1 https://github.com/stamparm/maltrail.git
cd maltrail
sudo python3 sensor.py

• For SUSE/openSUSE

sudo zypper install gcc gcc-c++ git libpcap-devel python3-devel python3-pip procps schedtool
sudo pip3 install pcapy-ng
git clone --depth 1 https://github.com/stamparm/maltrail.git
cd maltrail
sudo python3 sensor.py

Don't forget to put interfaces in promiscuous mode as needed:

for dev in $(ifconfig | grep mtu | grep -Eo '^\w+'); do ifconfig $dev promisc; done

To start the (optional) Server on same machine, open a new terminal and execute the following:

[[ -d maltrail ]] || git clone --depth 1 https://github.com/stamparm/maltrail.git
cd maltrail
python server.py

• For Docker

Currently only the server is available as a container image.

Start the container with docker run:

# Build image
# Start the server
docker run -d --name maltrail --restart=unless-stopped -p 8338:8338/tcp -p 8337:8337/udp -v /etc/maltrail.conf:/opt/maltrail/maltrail.conf:ro ghcr.io/stamparm/maltrail:latest
# Update the image regularly
docker stop maltrail
docker pull ghcr.io/stamparm/maltrail:latest
docker start maltrail

If you need a fixed version, change the docker run command to not start ghcr.io/stamparm/maltrail:latest but for example ghcr.io/stamparm/maltrail:0.84

... or with docker compose:

# For both
docker compose up -d
# Update image regularly
docker compose down --remove-orphans
docker compose build
docker compose up -d

Don't edit the docker-compose.yml file directly, as this will be overwritten by git pull. Instead, copy it to docker-compose.override.yml and edit that file; it is included in this repo's .gitignore.

To test that everything is up and running execute the following:

ping -c 1 136.161.101.53
cat /var/log/maltrail/$(date +"%Y-%m-%d").log

Also, to test the capturing of DNS traffic you can try the following:

nslookup morphed.ru
cat /var/log/maltrail/$(date +"%Y-%m-%d").log

To stop Sensor and Server instances (if running in background) execute the following:

sudo pkill -f sensor.py
pkill -f server.py

Access the reporting interface (i.e. Client) by visiting the http://127.0.0.1:8338 (default credentials: admin:changeme!) from your web browser:

Administrator's guide

Sensor

Sensor's configuration can be found inside the maltrail.conf file's section [Sensor]:

If option USE_MULTIPROCESSING is set to true then all CPU cores will be used. One core will be used only for packet capture (with appropriate affinity, IO priority and nice level settings), while other cores will be used for packet processing. Otherwise, everything will be run on a single core. Option USE_FEED_UPDATES can be used to turn off the trail updates from feeds altogether (and just use the provided static ones). Option UPDATE_PERIOD contains the number of seconds between each automatic trails update (Note: default value is set to 86400 (i.e. one day)) by using definitions inside the trails directory (Note: both Sensor and Server take care of the trails update). Option CUSTOM_TRAILS_DIR can be used by user to provide location of directory containing the custom trails (*.txt) files.

Option USE_HEURISTICS turns on heuristic mechanisms (e.g. long domain name (suspicious), excessive no such domain name (suspicious), direct .exe download (suspicious), etc.), potentially introducing false positives. Option CAPTURE_BUFFER presents a