pfSense fail2ban ipthreat integration

Discuss Wi-Fi setups, cybersecurity, and network troubleshooting.
Post Reply
User avatar
ccb056
Site Administrator
Posts: 981
Joined: January 14th, 2004, 11:36 pm
Location: Texas
Contact:

pfSense fail2ban ipthreat integration

Post by ccb056 »

This comprehensive guide provides step-by-step instructions for configuring a robust security monitoring system using FreeBSD syslogd to collect pfSense SSH logs, fail2ban for intrusion detection, and ipthreat.net for threat intelligence sharing.

Prerequisites
  • FreeBSD 14.3 server with root access
  • pfSense firewall with SSH enabled
  • Network connectivity between pfSense and FreeBSD server
  • Basic understanding of FreeBSD system administration
Phase 1: FreeBSD syslogd Configuration for Remote Logging

Configure syslogd for Remote Log Reception

Enable syslogd and configure remote logging in /etc/rc.conf:

Code: Select all

# Enable syslogd service
sysrc syslogd_enable="YES"

# Configure syslogd flags to accept logs from pfSense
sysrc syslogd_flags="-a 192.168.11.1:* -vv"  # Replace with your pfSense IP
Note:
  • The :* after the IP address allows connections from any source port
  • If syslogd was previously configured with -s (secure mode), it will not accept remote logs
  • You can verify the change worked:

Code: Select all

sysrc syslogd_flags
# Should show: syslogd_flags: -a 192.168.11.1:* -vv
Key parameters:
  • -a 192.168.11.1:*: Allow connections from this IP using any source port
  • -vv: Increase verbosity for troubleshooting (can be changed to -v or removed once working)
  • -s: Secure mode (blocks network connections - do NOT use for remote logging)
Configure Log File Destinations

Edit /etc/syslog.conf to separate pfSense logs:

Add these lines after the !* line and before the include directives:

Code: Select all

+hostname.local.example.com
#
#       Spaces ARE valid field separators in this file. However,
#       other *nix-like systems still insist on using tabs as field
#       separators. If you are sharing this file between systems, you
#       may want to use only tabs as field separators here.
#       Consult the syslog.conf(5) manpage.
#
# ... (default rules for local host) ...
!*
+router.local.example.com
*.*                                     /var/log/pfsense.log
include                                 /etc/syslog.d
include                                 /usr/local/etc/syslog.d
Key points:
  • The +hostname.local.example.com at the top restricts the default rules to your local host only
  • The !* directive resets any previous program filters
  • +router.local.example.com matches logs from your pfSense hostname
  • *.* captures ALL logs from pfSense in /var/log/pfsense.log
  • This setup keeps local logs separate from pfSense logs
This configuration ensures that:
  1. Your FreeBSD host logs go to their normal locations
  2. pfSense logs are isolated in their own file
  3. No mixing of local and remote logs occurs
Create Log File and Set Permissions

Code: Select all

# Create the log file
touch /var/log/pfsense.log
Start and Test syslogd

Code: Select all

# Restart syslogd
service syslogd restart

# Verify syslogd is listening
netstat -an | grep :514
sockstat -4 -l | grep :514

# Test local logging
logger -p auth.info "Test message from FreeBSD"
grep "Test message" /var/log/auth.log

# Test remote logging from pfSense
# SSH to pfSense and run this command:
logger -p auth.info "Test remote message from pfSense"

# Verify pfSense logs are arriving
tail -f /var/log/pfsense.log

# If no logs appear, check if pfSense is sending traffic
tcpdump -i lagg0 -n 'udp port 514 and host 192.168.11.1'

# To see the actual syslog message payloads
tcpdump -i lagg0 -n -A 'udp port 514 and host 192.168.11.1'

# For more detailed packet inspection with timestamps
tcpdump -i lagg0 -n -A -s0 -vv 'udp port 514 and host 192.168.11.1'

# Check syslog.conf formatting (tabs shown as \011)
grep -A2 "+router" /etc/syslog.conf | vis -t
# Expected output should show tabs as \011 between selectors and file paths:
# +router.local.example.com
# *.*\011\011\011\011\011\011\011\011/var/log/pfsense.log
Phase 2: pfSense Configuration for SSH Log Forwarding

Access pfSense Web Interface

Navigate to: Status → System Logs → SettingsRemote Logging Options

Configure Remote Logging Settings

Based on your pfSense interface, configure as follows:
  1. Enable Remote Logging: ✓ Check "Send log messages to remote syslog server"
  2. Source Address: LAN - Select your LAN interface (or Default (any) if preferred)
  3. IP Protocol: IPv4 - Already correctly selected
  4. Remote log servers:
    - First field: 192.168.11.11 (your FreeBSD server IP)
    - Port field: 514 (standard syslog port)
  5. Remote Syslog Contents: Select ONLY what you need for SSH monitoring:
    - [ ] Everything (unchecked - too much data)
    - [ ] System Events (unchecked - not needed for SSH)
    - [ ] Firewall Events (optional - only if you want to track firewall blocks)
    - [ ] DNS Events (unchecked)
    - [ ] DHCP Events (unchecked)
    - [ ] PPP Events (unchecked)
    - [✓] General Authentication Events (CHECKED - this contains SSH login attempts)
    - [ ] Captive Portal Events (unchecked)
    - [ ] VPN Events (unchecked)
    - [ ] Gateway Monitor Events (unchecked)
    - [ ] Routing Daemon Events (unchecked)
    - [ ] Network Time Protocol Events (unchecked)
    - [ ] Wireless Events (unchecked)
The General Authentication Events option is the key setting - it includes all SSH authentication attempts from the auth.log file.

Save Configuration

Click Save at the bottom of the page to apply the settings.

Test pfSense Log Forwarding

Code: Select all

# On FreeBSD server, monitor logs in real-time
tail -f /var/log/pfsense.log

# Generate test SSH failure on pfSense (from another machine)
ssh invalid_user@pfsense_ip

# Verify logs appear in FreeBSD
grep "authentication failure" /var/log/pfsense.log
Phase 3: fail2ban Installation and Configuration

Install fail2ban on FreeBSD

Code: Select all

# Search for available fail2ban packages
pkg search fail2ban

# Install the appropriate Python 3.11 version
pkg install py311-fail2ban

# Install curl (required for ipthreat.net API calls)
pkg install curl

# The installation will include dependencies:
# - python311
# - py311-pyinotify (for file monitoring)
# - py311-sqlite3 (for ban database)
# - libinotify (kernel file monitoring support)
Post-Installation Configuration

Based on the installation messages, note these important points:
  1. DO NOT edit the default configuration files (fail2ban.conf, jail.conf) as they will be overwritten on upgrades
  2. Create .local files for all customizations (e.g., jail.local, fail2ban.local)
Enable fail2ban Service

Code: Select all

# Enable in rc.conf
sysrc fail2ban_enable="YES"
Configure fail2ban for pfSense SSH Logs

Create or edit /usr/local/etc/fail2ban/jail.local:

Code: Select all

[INCLUDES]
before = paths-freebsd.conf

[DEFAULT]
# ipthreat.net integration (reporting plus local firewall action)
action_ipthreat = ipthreat
action = %(action_ipthreat)s[]

# "bantime" is the amount of time that a host is banned, integer in seconds or
# time abbreviation format (m - minutes, h - hours, d - days, w - weeks, mo - months, y - years).
# This is to consider as an initial time if bantime.increment gets enabled.
bantime  = 7d

# A host is banned if it has generated "maxretry" during the last "findtime"
# seconds. Can use time abbreviation format (m - minutes, h - hours, d - days, w - weeks, mo - months, y - years).
findtime = 7d

[sshd]
enabled = true
# To use more aggressive sshd modes set filter parameter "mode" in jail.local:
# normal (default), ddos, extra or aggressive (combines all).
# See "tests/files/logs/sshd" or "filter.d/sshd.conf" for usage example and details.
#mode   = normal
port    = ssh
logpath = /var/log/pfsense.log
backend = %(sshd_backend)s
maxretry = 2

[sshguard-attack]
enabled = true
filter = sshguard-attack
logpath = /var/log/pfsense.log
backend = auto
maxretry = 2
# Override the 'system' field to be 'sshd'
action = %(action_ipthreat)s[ipthreat_system="sshd"]

[sshguard-block]
enabled = true
filter = sshguard-block
logpath = /var/log/pfsense.log
backend = auto
maxretry = 1
# Override the 'system' field to be 'sshd'
action = %(action_ipthreat)s[ipthreat_system="sshd"]
This configuration will:
  • sshd jail: Monitor SSH login failures and report after 2 failed attempts
  • sshguard-attack jail: Monitor sshguard attack detections and report after 2 attacks
  • sshguard-block jail: Monitor sshguard blocking decisions and report immediately (maxretry=1)
  • All jails only report to ipthreat.net (no local firewall actions)
  • Provide comprehensive coverage of SSH failures, attacks, and confirmed blocks
Create Custom Filters for pfSense sshguard Logs

Create /usr/local/etc/fail2ban/filter.d/sshguard-attack.conf:

Code: Select all

# Fail2Ban filter for sshguard attacks from pfSense
#
# Monitors sshguard attack detections only
#

[INCLUDES]
before = common.conf

[Definition]

_daemon = sshguard

# Option:  failregex
# Notes.:  regex to match sshguard attack messages only
# Values:  TEXT
failregex = ^Attack\s+from\s+"<HOST>"\s+on\s+service\s+SSH\s+with\s+danger\s+\d+\.\s*$

# Option: prefregex
# Notes.:  Regex to match the log line prefix.
# Values:  TEXT
prefregex = ^<F-MLFID>%(__prefix_line)s</F-MLFID>\s*<F-CONTENT>.+</F-CONTENT>$

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
ignoreregex =
Create /usr/local/etc/fail2ban/filter.d/sshguard-block.conf:

Code: Select all

# Fail2Ban filter for sshguard blocks from pfSense
#
# Monitors sshguard blocking decisions only
#

[INCLUDES]
before = common.conf

[Definition]

_daemon = sshguard

# Option:  failregex
# Notes.:  regex to match sshguard blocking messages only
# Values:  TEXT
failregex = ^Blocking\s+"<HOST>/\d+"\s+for\s+\d+\s+secs.*$

# Option: prefregex
# Notes.:  Regex to match the log line prefix.
# Values:  TEXT
prefregex = ^<F-MLFID>%(__prefix_line)s</F-MLFID>\s*<F-CONTENT>.+</F-CONTENT>$

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
ignoreregex =
These filters will:
  • sshguard-attack: Trigger on sshguard attack detections, report suspicious activity after 2 detections
  • sshguard-block: Trigger on sshguard blocking decisions, report confirmed blocks immediately
  • Extract the IP address from both attack and blocking messages
  • Report each detected IP to ipthreat.net for threat intelligence
Test Filter Configuration

Code: Select all

# Create a test log file with actual sshguard blocking entries
cat > /tmp/test-sshguard.log << 'EOF'
Aug  1 21:34:06 <auth.notice> router.local.example.com sshguard[87758]: Attack from "221.211.246.37" on service SSH with danger 2.
Aug  1 21:34:40 <auth.notice> router.local.example.com sshguard[87758]: Attack from "221.211.246.37" on service SSH with danger 10.
Aug  1 21:34:46 <auth.info> router.local.example.com sshguard[87758]: Blocking "221.211.246.37/32" for 2592000 secs (6 attacks in 40 secs, after 1 abuses over 40 secs.)
Aug  1 21:46:13 <auth.notice> router.local.example.com sshguard[87758]: Attack from "143.198.27.218" on service SSH with danger 2.
Aug  1 21:46:18 <auth.info> router.local.example.com sshguard[87758]: Blocking "143.198.27.218/32" for 2592000 secs (15 attacks in 5 secs, after 1 abuses over 5 secs.)
EOF

# Test the attack filter
fail2ban-regex /tmp/test-sshguard.log /usr/local/etc/fail2ban/filter.d/sshguard-attack.conf

# Test the block filter
fail2ban-regex /tmp/test-sshguard.log /usr/local/etc/fail2ban/filter.d/sshguard-block.conf

# Test against the actual log file once it has data
fail2ban-regex /var/log/pfsense.log /usr/local/etc/fail2ban/filter.d/sshguard-attack.conf
fail2ban-regex /var/log/pfsense.log /usr/local/etc/fail2ban/filter.d/sshguard-block.conf

# Test the built-in sshd filter as well
fail2ban-regex /var/log/pfsense.log sshd
Phase 4: ipthreat.net Integration Configuration

Register with ipthreat.net
  1. Create account at https://ipthreat.net/account/signup
  2. Verify email address and complete registration
  3. Navigate to Account → API Keys
  4. Generate new API key for fail2ban integration
Configure ipthreat.net Integration

Edit /usr/local/etc/fail2ban/action.d/ipthreat.conf and add your API key:

Code: Select all

[Init]
# Option:  ipthreat_apikey
# Notes    Your API key from ipthreat.net
# Values:  STRING Default: None
# Register for ipthreat [https://ipthreat.net], get api key and set below.
# You will need to set the flags and system in the action call in jail.conf
ipthreat_apikey = YOUR_ACTUAL_API_KEY_HERE
Phase 5: Service Startup and Testing

Start All Services

Code: Select all

# Start syslogd (should already be running)
service syslogd start

# Start fail2ban
service fail2ban start

# Verify services are running
service syslogd status
service fail2ban status
Comprehensive System Testing

Test 1: Log Reception

Code: Select all

# Monitor pfSense logs in real-time
tail -f /var/log/pfsense.log

# Look for sshguard messages
grep "sshguard" /var/log/pfsense.log | tail -10
Test 2: fail2ban Detection

Code: Select all

# Check overall fail2ban status
fail2ban-client status
# Expected output:
# Status
# |- Number of jail:      3
# `- Jail list:   sshd, sshguard-attack, sshguard-block

# Check detailed jail status for SSH failures
fail2ban-client status sshd

# Check detailed jail status for sshguard attacks
fail2ban-client status sshguard-attack

# Check detailed jail status for sshguard blocks  
fail2ban-client status sshguard-block
Test 3: ipthreat.net Reporting

Code: Select all

# Check ipthreat reporting in logs
grep -i "ipthreat\|curl" /var/log/fail2ban.log | tail -10

# Look for successful ban actions (reports to ipthreat)
grep "Ban " /var/log/fail2ban.log | tail -5

# Check for curl errors
grep "curl.*not found" /var/log/fail2ban.log
Phase 6: Log Rotation and Monitoring

Configure Log Rotation

Add to /etc/newsyslog.conf before the <include> directives:

Code: Select all

# pfSense and fail2ban log rotation
/var/log/pfsense.log            root:wheel  640  7   *  @T00  N
/var/log/fail2ban.log           root:wheel  644  7   *  @T00  N
Parameters explained:
  • 640/644: File permissions
  • 7: Keep 7 rotated copies for both logs
  • *: No size limit (time-based rotation only)
  • @T00: Rotate at midnight for both logs
  • N: No signal to syslogd (appropriate for non-syslogd managed logs), no compression
Test Log Rotation

Code: Select all

# Test rotation manually (dry run to check configuration)
newsyslog -nv

# Force rotation of all logs
newsyslog -F

# Manually rotate specific logs
newsyslog /var/log/pfsense.log
newsyslog /var/log/fail2ban.log
Real-time Monitoring

Code: Select all

# Watch fail2ban activity
tail -f /var/log/fail2ban.log

# Monitor SSH failures from pfSense
tail -f /var/log/pfsense.log | grep --color=always "sshd"

# Check recent ipthreat.net reports
grep "ipthreat.net/report" /var/log/fail2ban.log | tail -20

# Quick status check
fail2ban-client status sshd
Troubleshooting Common Issues

syslogd Issues

Problem: No logs received from pfSense

Code: Select all

# Check syslogd is listening
netstat -an | grep :514

# Test with verbose logging
sysrc syslogd_flags="-a 192.168.11.1:* -d -v -v"  
service syslogd restart

# Verify hostname in syslog.conf matches pfSense
grep "+router" /etc/syslog.conf
Problem: Wrong log file destinations

Code: Select all

# Verify syslog.conf syntax
syslogd -t -f /etc/syslog.conf

# Check file permissions
ls -la /var/log/pfsense*.log
Common fail2ban Issues

Problem: curl not found error

Code: Select all

# Error: '/bin/sh: curl: not found'
# Solution: Install curl
pkg install curl

# Verify curl is in PATH
which curl
/usr/local/bin/curl

# Restart fail2ban after installing curl
service fail2ban restart

# Check if the PATH issue is resolved
grep -a PATH= /proc/`pidof -x fail2ban-server`/environ
Problem: fail2ban not starting

Code: Select all

# Test configuration
fail2ban-server -t

# Check for errors
tail -f /var/log/fail2ban.log

# Verify Python dependencies
pkg info | grep py311-fail2ban

# Ensure curl is installed
which curl || pkg install curl
Problem: IPs not getting reported

Code: Select all

# Test both filter regexes
fail2ban-regex /var/log/pfsense.log /usr/local/etc/fail2ban/filter.d/sshguard-attack.conf
fail2ban-regex /var/log/pfsense.log /usr/local/etc/fail2ban/filter.d/sshguard-block.conf
fail2ban-regex /var/log/pfsense.log sshd

# Check jail configurations
fail2ban-client get sshd logpath
fail2ban-client get sshd maxretry
fail2ban-client get sshd findtime

fail2ban-client get sshguard-attack logpath
fail2ban-client get sshguard-attack maxretry
fail2ban-client get sshguard-attack findtime

fail2ban-client get sshguard-block logpath
fail2ban-client get sshguard-block maxretry
fail2ban-client get sshguard-block findtime
ipthreat.net Issues

Problem: ipthreat.net API calls failing

Code: Select all

# Test curl is working
which curl
curl --version

# Test API manually (replace YOUR_API_KEY)
curl -sSf "https://api.ipthreat.net/api/report" \
     -X POST \
     -H "Content-Type: application/json" \
     -H "X-API-KEY: YOUR_API_KEY" \
     -d '{"ip":"192.0.2.1","flags":"8","system":"test","notes":"manual test"}'

# Check fail2ban action logs
grep "ipthreat" /var/log/fail2ban.log | tail -10

# Look for successful reports
grep "Ban " /var/log/fail2ban.log | tail -5

# Check for curl errors specifically
grep "curl.*not found" /var/log/fail2ban.log
Security Considerations
  1. Network Security: Use dedicated management VLANs for syslog traffic if possible
  2. API Keys: Store ipthreat.net API keys securely, rotate regularly
  3. Log Retention: Balance security needs with storage constraints
  4. Access Control: Restrict access to log files and configuration
  5. Monitoring: Implement alerting for service failures
  6. Updates: Keep fail2ban and FreeBSD updated for security patches
Summary

This configuration provides a streamlined security monitoring system that:
  • Collects SSH security events from pfSense via syslog
  • Monitors SSH authentication failures directly from pfSense logs
  • Detects sshguard attack patterns and blocking decisions
  • Reports confirmed attackers to the ipthreat.net threat intelligence database
  • Requires no local firewall modifications
  • Minimizes false positives by using different thresholds for different event types
The system operates passively, contributing to the global threat intelligence community without impacting your local network security policies.
Post Reply