Beginning to receive logs from pfSense/Suricata and noticed something was off. When trying to parse the JSON messages, very few messages would actually pass. Nothing wrong with Graylog JSON parser, but simply that pfSense syslog daemon will automatically truncate exported messages to 480 bytes max. 480 bytes is a fair bit, but not enough when outputting in JSON format from Suricata. So this is a bit marginal use case; exporting JSON formatted logs from Suricata in pfSense :)
This is just a note-to-self and may include errors and misunderstandings.
The problematic log will look something like this:
{"timestamp":"2022-02-15T16:47:47.856598+0100","flow_id":1034522015285799,"in_iface":"igb0","event_type":"dns", "src_ip":"X.X.X.X", "src_port":21295, "dest_ip":"X.X.X.X","dest_port":53,"proto":"UDP","dns": {"version":2,"type":"answer", "id":50199,"flags":"8180", "qr":true,"rd":true,"ra":true,"rrname":"signaler-pa.clients6.google.com","rrtype":"A","rcode":"NOERROR","answers":[{"rrname":"signaler-
Something is obviously wrong here. An insane amount of RegEx could possible solve this, but decided against it. Instead going for a workaround with syslog-ng: From Suricata output to file and use syslog-ng to send these logs to Graylog instead. Should get the full message this way. Syslog-ng is available from pfSense package manager.
1) Service > Suricata
Edit the relevant interface. In 'EVE JSON Settings' choose 'FILE' in 'EVE Output Type'.
You can still choose to copy Suricata messages to the firewall system log (in general settings). These messages will not include the full (or partly full) JSON messages – these logs are exported with syslog-ng. You properly still want to export pfSense logs anyway.
2) pfSense > Diagnostics > Command prompt >
ls -l /var/log/suricata/
Look for a suricata_igbXXXXXXX directory. Properly more than one.
ls -l /var/log/suricata/suricata_igbXXXXXXX
Should get you (among other) an 'eve.json' file.
Have a look-see (it is a lot of data)
cat /var/log/suricata/suricata_igb0XXXXXXX/eve.json
Notice all lines are well formatted.
3) pfSense > Services > Syslog-ng > Advanced
Like all other syslog-ng setups, you need three parts: 1) destination. 2) Source and 3) Log (You can do a lot more with syslog-ng. This is the absolute basics.)
4) pfSense > Services > Syslog-ng > Advanced | Destination
Object Name: Suricata
Object Type: Destination
Object Parameters:
{
udp("192.168.10.101" port(1515));
};
192.168.10.101 is IP of log-server. 1515 is the port.
5) pfSense > Services > Syslog-ng > Advanced | Source
Object Name: Suricata
Object Type: Source
Object Parameters:
{
wildcard-file(
base-dir("/var/log/suricata")
filename-pattern("eve.json")
recursive(yes)
follow-freq(1)
program-override("suricata")
flags(no-parse)
);
};
follow-freq(1): Checks the eve.json file every second for new messages.
6) pfSense > Services > Syslog-ng > Advanced | Log
Object Name: Suricata
Object Type: Log
Object Parameters:
{
source(Suricata);
destination(Suricata);
};
Source and destination is the two above objects mentioned in 5) and 6).
7) pfSense > Services > Syslog-ng
Choose interface and enable the syslog-ng service. If your Graylog server is running on IP 192.168.10.101 and you have enabled a Syslog UPD input on port 1515, full logs messages should start to arrive.